2023-02-08 23:53:29,085 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-08 23:53:29,359 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d 2023-02-08 23:53:29,375 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-08 23:53:29,375 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/cluster_0361639e-2335-beea-2849-07295ee226f0, deleteOnExit=true 2023-02-08 23:53:29,376 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-08 23:53:29,376 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/test.cache.data in system properties and HBase conf 2023-02-08 23:53:29,377 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/hadoop.tmp.dir in system properties and HBase conf 2023-02-08 23:53:29,377 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/hadoop.log.dir in system properties and HBase conf 2023-02-08 23:53:29,378 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-08 23:53:29,378 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-08 23:53:29,378 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-08 23:53:29,497 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-08 23:53:29,918 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-08 23:53:29,922 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-08 23:53:29,922 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-08 23:53:29,922 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-08 23:53:29,923 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 23:53:29,923 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-08 23:53:29,923 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-08 23:53:29,923 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 23:53:29,924 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 23:53:29,924 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-08 23:53:29,924 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/nfs.dump.dir in system properties and HBase conf 2023-02-08 23:53:29,925 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/java.io.tmpdir in system properties and HBase conf 2023-02-08 23:53:29,925 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 23:53:29,925 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-08 23:53:29,925 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-08 23:53:30,475 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 23:53:30,479 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 23:53:30,792 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-08 23:53:30,958 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-08 23:53:30,978 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 23:53:31,021 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 23:53:31,069 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/java.io.tmpdir/Jetty_localhost_41351_hdfs____.hkrdno/webapp 2023-02-08 23:53:31,248 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41351 2023-02-08 23:53:31,266 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 23:53:31,267 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 23:53:31,811 WARN [Listener at localhost/36245] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 23:53:31,875 WARN [Listener at localhost/36245] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 23:53:31,892 WARN [Listener at localhost/36245] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 23:53:31,898 INFO [Listener at localhost/36245] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 23:53:31,902 INFO [Listener at localhost/36245] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/java.io.tmpdir/Jetty_localhost_45989_datanode____dq03ls/webapp 2023-02-08 23:53:31,996 INFO [Listener at localhost/36245] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45989 2023-02-08 23:53:32,264 WARN [Listener at localhost/41521] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 23:53:32,698 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x70474341f2f386b5: Processing first storage report for DS-61f86713-7f5b-4180-a47d-d3d547746358 from datanode 837303ae-269b-4144-a273-fa40483f1512 2023-02-08 23:53:32,700 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x70474341f2f386b5: from storage DS-61f86713-7f5b-4180-a47d-d3d547746358 node DatanodeRegistration(127.0.0.1:42851, datanodeUuid=837303ae-269b-4144-a273-fa40483f1512, infoPort=40221, infoSecurePort=0, ipcPort=41521, storageInfo=lv=-57;cid=testClusterID;nsid=97279396;c=1675900410558), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-08 23:53:32,700 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x70474341f2f386b5: Processing first storage report for DS-da9e2cc1-48e0-4cfb-ae56-83952fea5428 from datanode 837303ae-269b-4144-a273-fa40483f1512 2023-02-08 23:53:32,700 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x70474341f2f386b5: from storage DS-da9e2cc1-48e0-4cfb-ae56-83952fea5428 node DatanodeRegistration(127.0.0.1:42851, datanodeUuid=837303ae-269b-4144-a273-fa40483f1512, infoPort=40221, infoSecurePort=0, ipcPort=41521, storageInfo=lv=-57;cid=testClusterID;nsid=97279396;c=1675900410558), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 23:53:32,758 DEBUG [Listener at localhost/41521] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d 2023-02-08 23:53:32,816 INFO [Listener at localhost/41521] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/cluster_0361639e-2335-beea-2849-07295ee226f0/zookeeper_0, clientPort=54774, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/cluster_0361639e-2335-beea-2849-07295ee226f0/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/cluster_0361639e-2335-beea-2849-07295ee226f0/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-08 23:53:32,829 INFO [Listener at localhost/41521] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=54774 2023-02-08 23:53:32,836 INFO [Listener at localhost/41521] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:32,839 INFO [Listener at localhost/41521] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:33,482 INFO [Listener at localhost/41521] util.FSUtils(479): Created version file at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7 with version=8 2023-02-08 23:53:33,483 INFO [Listener at localhost/41521] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/hbase-staging 2023-02-08 23:53:33,798 INFO [Listener at localhost/41521] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-08 23:53:34,255 INFO [Listener at localhost/41521] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-02-08 23:53:34,286 INFO [Listener at localhost/41521] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:53:34,287 INFO [Listener at localhost/41521] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 23:53:34,287 INFO [Listener at localhost/41521] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 23:53:34,287 INFO [Listener at localhost/41521] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:53:34,287 INFO [Listener at localhost/41521] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 23:53:34,423 INFO [Listener at localhost/41521] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 23:53:34,492 DEBUG [Listener at localhost/41521] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-08 23:53:34,583 INFO [Listener at localhost/41521] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39489 2023-02-08 23:53:34,593 INFO [Listener at localhost/41521] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:34,595 INFO [Listener at localhost/41521] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:34,614 INFO [Listener at localhost/41521] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39489 connecting to ZooKeeper ensemble=127.0.0.1:54774 2023-02-08 23:53:34,655 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:394890x0, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 23:53:34,657 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:39489-0x100365acd470000 connected 2023-02-08 23:53:34,682 DEBUG [Listener at localhost/41521] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 23:53:34,683 DEBUG [Listener at localhost/41521] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:53:34,686 DEBUG [Listener at localhost/41521] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 23:53:34,693 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39489 2023-02-08 23:53:34,694 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39489 2023-02-08 23:53:34,694 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39489 2023-02-08 23:53:34,695 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39489 2023-02-08 23:53:34,695 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39489 2023-02-08 23:53:34,701 INFO [Listener at localhost/41521] master.HMaster(439): hbase.rootdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7, hbase.cluster.distributed=false 2023-02-08 23:53:34,768 INFO [Listener at localhost/41521] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-02-08 23:53:34,769 INFO [Listener at localhost/41521] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:53:34,769 INFO [Listener at localhost/41521] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 23:53:34,769 INFO [Listener at localhost/41521] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 23:53:34,769 INFO [Listener at localhost/41521] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 23:53:34,769 INFO [Listener at localhost/41521] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 23:53:34,774 INFO [Listener at localhost/41521] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 23:53:34,776 INFO [Listener at localhost/41521] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42703 2023-02-08 23:53:34,778 INFO [Listener at localhost/41521] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 23:53:34,784 DEBUG [Listener at localhost/41521] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 23:53:34,785 INFO [Listener at localhost/41521] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:34,787 INFO [Listener at localhost/41521] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:34,789 INFO [Listener at localhost/41521] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42703 connecting to ZooKeeper ensemble=127.0.0.1:54774 2023-02-08 23:53:34,792 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:427030x0, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 23:53:34,793 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:42703-0x100365acd470001 connected 2023-02-08 23:53:34,793 DEBUG [Listener at localhost/41521] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 23:53:34,794 DEBUG [Listener at localhost/41521] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:53:34,795 DEBUG [Listener at localhost/41521] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 23:53:34,795 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42703 2023-02-08 23:53:34,796 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42703 2023-02-08 23:53:34,796 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42703 2023-02-08 23:53:34,796 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42703 2023-02-08 23:53:34,797 DEBUG [Listener at localhost/41521] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42703 2023-02-08 23:53:34,798 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:53:34,807 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 23:53:34,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:53:34,824 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 23:53:34,825 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 23:53:34,825 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:53:34,826 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 23:53:34,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,39489,1675900413627 from backup master directory 2023-02-08 23:53:34,827 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 23:53:34,836 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:53:34,836 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 23:53:34,836 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 23:53:34,837 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:53:34,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-08 23:53:34,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-08 23:53:34,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/hbase.id with ID: e7332626-208e-4c2d-baae-1ddac3e0aab3 2023-02-08 23:53:34,958 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:34,972 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:53:35,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x0b955e48 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:53:35,039 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@795cd6dd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:53:35,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-08 23:53:35,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-08 23:53:35,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-08 23:53:35,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-08 23:53:35,083 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:53:35,087 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:53:35,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 23:53:35,120 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/data/master/store-tmp 2023-02-08 23:53:35,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:35,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 23:53:35,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:53:35,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:53:35,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 23:53:35,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:53:35,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:53:35,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675900415146Disabling compacts and flushes for region at 1675900415146Disabling writes for close at 1675900415147 (+1 ms)Writing region close event to WAL at 1675900415147Closed at 1675900415147 2023-02-08 23:53:35,148 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/WALs/jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:53:35,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39489%2C1675900413627, suffix=, logDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/WALs/jenkins-hbase4.apache.org,39489,1675900413627, archiveDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/oldWALs, maxLogs=10 2023-02-08 23:53:35,217 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42851,DS-61f86713-7f5b-4180-a47d-d3d547746358,DISK] 2023-02-08 23:53:35,224 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-08 23:53:35,280 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/WALs/jenkins-hbase4.apache.org,39489,1675900413627/jenkins-hbase4.apache.org%2C39489%2C1675900413627.1675900415177 2023-02-08 23:53:35,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42851,DS-61f86713-7f5b-4180-a47d-d3d547746358,DISK]] 2023-02-08 23:53:35,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:53:35,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:35,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:53:35,285 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:53:35,341 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:53:35,347 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-08 23:53:35,369 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-08 23:53:35,381 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:35,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:53:35,389 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:53:35,404 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 23:53:35,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:53:35,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72360674, jitterRate=0.07825806736946106}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:53:35,410 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675900415310Initializing all the Stores at 1675900415313 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900415313Cleaning up temporary data from old regions at 1675900415395 (+82 ms)Cleaning up detritus from prior splits at 1675900415396 (+1 ms)Region opened successfully at 1675900415410 (+14 ms) 2023-02-08 23:53:35,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-08 23:53:35,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-08 23:53:35,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-08 23:53:35,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 23:53:35,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 23:53:35,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-08 23:53:35,465 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-08 23:53:35,489 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-08 23:53:35,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-08 23:53:35,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-08 23:53:35,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-08 23:53:35,522 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-08 23:53:35,527 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-08 23:53:35,531 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-08 23:53:35,534 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:53:35,535 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-08 23:53:35,536 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-08 23:53:35,547 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-08 23:53:35,552 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 23:53:35,552 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 23:53:35,552 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:53:35,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,39489,1675900413627, sessionid=0x100365acd470000, setting cluster-up flag (Was=false) 2023-02-08 23:53:35,566 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:53:35,572 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-08 23:53:35,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:53:35,578 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:53:35,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-08 23:53:35,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:53:35,588 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/.tmp 2023-02-08 23:53:35,600 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(952): ClusterId : e7332626-208e-4c2d-baae-1ddac3e0aab3 2023-02-08 23:53:35,605 DEBUG [RS:0;jenkins-hbase4:42703] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 23:53:35,610 DEBUG [RS:0;jenkins-hbase4:42703] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 23:53:35,611 DEBUG [RS:0;jenkins-hbase4:42703] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 23:53:35,614 DEBUG [RS:0;jenkins-hbase4:42703] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 23:53:35,615 DEBUG [RS:0;jenkins-hbase4:42703] zookeeper.ReadOnlyZKClient(139): Connect 0x01e7a4d7 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:53:35,623 DEBUG [RS:0;jenkins-hbase4:42703] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@f205a9b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:53:35,624 DEBUG [RS:0;jenkins-hbase4:42703] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@b1fcfa7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 23:53:35,656 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42703 2023-02-08 23:53:35,661 INFO [RS:0;jenkins-hbase4:42703] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 23:53:35,661 INFO [RS:0;jenkins-hbase4:42703] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 23:53:35,661 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 23:53:35,665 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,39489,1675900413627 with isa=jenkins-hbase4.apache.org/172.31.14.131:42703, startcode=1675900414768 2023-02-08 23:53:35,683 DEBUG [RS:0;jenkins-hbase4:42703] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 23:53:35,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-08 23:53:35,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 23:53:35,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 23:53:35,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 23:53:35,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 23:53:35,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-08 23:53:35,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 23:53:35,710 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675900445712 2023-02-08 23:53:35,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-08 23:53:35,722 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 23:53:35,723 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-08 23:53:35,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-08 23:53:35,730 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 23:53:35,735 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57749, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 23:53:35,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-08 23:53:35,742 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-08 23:53:35,742 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-08 23:53:35,742 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-08 23:53:35,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,745 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-08 23:53:35,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-08 23:53:35,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-08 23:53:35,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-08 23:53:35,749 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-08 23:53:35,749 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-08 23:53:35,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675900415750,5,FailOnTimeoutGroup] 2023-02-08 23:53:35,751 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675900415750,5,FailOnTimeoutGroup] 2023-02-08 23:53:35,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-08 23:53:35,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,776 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(2829): Master is not running yet 2023-02-08 23:53:35,776 WARN [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-08 23:53:35,877 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,39489,1675900413627 with isa=jenkins-hbase4.apache.org/172.31.14.131:42703, startcode=1675900414768 2023-02-08 23:53:35,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:35,888 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7 2023-02-08 23:53:35,889 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:36245 2023-02-08 23:53:35,889 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 23:53:35,893 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 23:53:35,894 DEBUG [RS:0;jenkins-hbase4:42703] zookeeper.ZKUtil(162): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:35,894 WARN [RS:0;jenkins-hbase4:42703] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 23:53:35,894 INFO [RS:0;jenkins-hbase4:42703] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 23:53:35,896 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1947): logDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/WALs/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:35,896 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42703,1675900414768] 2023-02-08 23:53:35,904 DEBUG [RS:0;jenkins-hbase4:42703] zookeeper.ZKUtil(162): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:35,914 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 23:53:35,921 INFO [RS:0;jenkins-hbase4:42703] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 23:53:35,940 INFO [RS:0;jenkins-hbase4:42703] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 23:53:35,943 INFO [RS:0;jenkins-hbase4:42703] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:53:35,943 INFO [RS:0;jenkins-hbase4:42703] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,944 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 23:53:35,950 INFO [RS:0;jenkins-hbase4:42703] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,951 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,951 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,951 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,951 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,952 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,952 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 23:53:35,952 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,952 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,952 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,952 DEBUG [RS:0;jenkins-hbase4:42703] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 23:53:35,953 INFO [RS:0;jenkins-hbase4:42703] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,953 INFO [RS:0;jenkins-hbase4:42703] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,953 INFO [RS:0;jenkins-hbase4:42703] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,969 INFO [RS:0;jenkins-hbase4:42703] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 23:53:35,973 INFO [RS:0;jenkins-hbase4:42703] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42703,1675900414768-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:35,988 INFO [RS:0;jenkins-hbase4:42703] regionserver.Replication(203): jenkins-hbase4.apache.org,42703,1675900414768 started 2023-02-08 23:53:35,989 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42703,1675900414768, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42703, sessionid=0x100365acd470001 2023-02-08 23:53:35,989 DEBUG [RS:0;jenkins-hbase4:42703] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 23:53:35,989 DEBUG [RS:0;jenkins-hbase4:42703] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:35,989 DEBUG [RS:0;jenkins-hbase4:42703] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42703,1675900414768' 2023-02-08 23:53:35,989 DEBUG [RS:0;jenkins-hbase4:42703] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:35,990 DEBUG [RS:0;jenkins-hbase4:42703] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:35,990 DEBUG [RS:0;jenkins-hbase4:42703] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 23:53:35,990 DEBUG [RS:0;jenkins-hbase4:42703] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 23:53:35,990 DEBUG [RS:0;jenkins-hbase4:42703] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:35,990 DEBUG [RS:0;jenkins-hbase4:42703] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42703,1675900414768' 2023-02-08 23:53:35,990 DEBUG [RS:0;jenkins-hbase4:42703] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 23:53:35,991 DEBUG [RS:0;jenkins-hbase4:42703] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 23:53:35,991 DEBUG [RS:0;jenkins-hbase4:42703] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 23:53:35,991 INFO [RS:0;jenkins-hbase4:42703] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 23:53:35,991 INFO [RS:0;jenkins-hbase4:42703] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 23:53:36,102 INFO [RS:0;jenkins-hbase4:42703] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42703%2C1675900414768, suffix=, logDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/WALs/jenkins-hbase4.apache.org,42703,1675900414768, archiveDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/oldWALs, maxLogs=32 2023-02-08 23:53:36,120 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42851,DS-61f86713-7f5b-4180-a47d-d3d547746358,DISK] 2023-02-08 23:53:36,123 INFO [RS:0;jenkins-hbase4:42703] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/WALs/jenkins-hbase4.apache.org,42703,1675900414768/jenkins-hbase4.apache.org%2C42703%2C1675900414768.1675900416104 2023-02-08 23:53:36,123 DEBUG [RS:0;jenkins-hbase4:42703] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42851,DS-61f86713-7f5b-4180-a47d-d3d547746358,DISK]] 2023-02-08 23:53:36,168 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 23:53:36,169 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 23:53:36,169 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7 2023-02-08 23:53:36,583 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:36,585 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 23:53:36,588 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/info 2023-02-08 23:53:36,589 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 23:53:36,590 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:36,590 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 23:53:36,593 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/rep_barrier 2023-02-08 23:53:36,594 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 23:53:36,595 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:36,595 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 23:53:36,597 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/table 2023-02-08 23:53:36,598 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 23:53:36,598 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:36,600 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740 2023-02-08 23:53:36,601 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740 2023-02-08 23:53:36,604 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 23:53:36,606 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 23:53:36,609 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:53:36,610 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68418413, jitterRate=0.019513800740242004}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 23:53:36,611 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675900416583Initializing all the Stores at 1675900416585 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900416585Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900416585Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900416585Cleaning up temporary data from old regions at 1675900416602 (+17 ms)Cleaning up detritus from prior splits at 1675900416603 (+1 ms)Region opened successfully at 1675900416611 (+8 ms) 2023-02-08 23:53:36,611 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 23:53:36,611 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 23:53:36,611 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 23:53:36,611 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 23:53:36,611 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 23:53:36,612 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 23:53:36,612 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675900416611Disabling compacts and flushes for region at 1675900416611Disabling writes for close at 1675900416611Writing region close event to WAL at 1675900416612 (+1 ms)Closed at 1675900416612 2023-02-08 23:53:36,617 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 23:53:36,617 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-08 23:53:36,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-08 23:53:36,636 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-08 23:53:36,639 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-08 23:53:36,790 DEBUG [jenkins-hbase4:39489] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-08 23:53:36,793 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42703,1675900414768, state=OPENING 2023-02-08 23:53:36,800 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-08 23:53:36,802 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:53:36,802 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 23:53:36,805 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:53:36,987 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:36,990 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 23:53:36,993 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36932, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 23:53:37,003 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-08 23:53:37,004 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 23:53:37,007 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42703%2C1675900414768.meta, suffix=.meta, logDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/WALs/jenkins-hbase4.apache.org,42703,1675900414768, archiveDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/oldWALs, maxLogs=32 2023-02-08 23:53:37,024 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42851,DS-61f86713-7f5b-4180-a47d-d3d547746358,DISK] 2023-02-08 23:53:37,028 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/WALs/jenkins-hbase4.apache.org,42703,1675900414768/jenkins-hbase4.apache.org%2C42703%2C1675900414768.meta.1675900417008.meta 2023-02-08 23:53:37,029 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42851,DS-61f86713-7f5b-4180-a47d-d3d547746358,DISK]] 2023-02-08 23:53:37,029 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:53:37,031 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-08 23:53:37,047 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-08 23:53:37,052 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-08 23:53:37,057 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-08 23:53:37,058 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:37,058 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-08 23:53:37,058 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-08 23:53:37,060 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 23:53:37,063 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/info 2023-02-08 23:53:37,063 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/info 2023-02-08 23:53:37,064 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 23:53:37,065 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:37,065 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 23:53:37,066 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/rep_barrier 2023-02-08 23:53:37,066 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/rep_barrier 2023-02-08 23:53:37,067 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 23:53:37,067 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:37,068 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 23:53:37,069 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/table 2023-02-08 23:53:37,069 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/table 2023-02-08 23:53:37,069 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 23:53:37,070 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:37,072 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740 2023-02-08 23:53:37,074 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740 2023-02-08 23:53:37,077 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 23:53:37,080 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 23:53:37,081 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65665524, jitterRate=-0.021507441997528076}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 23:53:37,081 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675900417058Writing region info on filesystem at 1675900417058Initializing all the Stores at 1675900417060 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900417060Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675900417060Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900417060Cleaning up temporary data from old regions at 1675900417075 (+15 ms)Cleaning up detritus from prior splits at 1675900417076 (+1 ms)Running coprocessor post-open hooks at 1675900417081 (+5 ms)Region opened successfully at 1675900417081 2023-02-08 23:53:37,090 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675900416980 2023-02-08 23:53:37,105 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-08 23:53:37,106 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-08 23:53:37,106 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42703,1675900414768, state=OPEN 2023-02-08 23:53:37,109 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-08 23:53:37,109 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 23:53:37,113 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-08 23:53:37,113 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42703,1675900414768 in 304 msec 2023-02-08 23:53:37,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-08 23:53:37,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 489 msec 2023-02-08 23:53:37,122 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.4850 sec 2023-02-08 23:53:37,122 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675900414821, completionTime=-1 2023-02-08 23:53:37,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-08 23:53:37,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 23:53:37,183 DEBUG [hconnection-0x6842f108-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:53:37,186 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36946, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:53:37,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-08 23:53:37,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675900477202 2023-02-08 23:53:37,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675900537202 2023-02-08 23:53:37,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 79 msec 2023-02-08 23:53:37,224 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39489,1675900413627-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:37,224 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39489,1675900413627-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:37,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39489,1675900413627-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:37,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39489, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:37,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-08 23:53:37,232 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-08 23:53:37,240 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-08 23:53:37,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 23:53:37,251 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-08 23:53:37,253 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:53:37,255 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:53:37,276 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:53:37,278 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684 empty. 2023-02-08 23:53:37,280 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:53:37,280 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-08 23:53:37,703 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-08 23:53:37,706 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 31ce00ef22047cfd9f24d7e09ef0a684, NAME => 'hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:53:38,118 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:38,118 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 31ce00ef22047cfd9f24d7e09ef0a684, disabling compactions & flushes 2023-02-08 23:53:38,118 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:53:38,118 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:53:38,118 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. after waiting 0 ms 2023-02-08 23:53:38,118 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:53:38,118 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:53:38,118 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 31ce00ef22047cfd9f24d7e09ef0a684: Waiting for close lock at 1675900418118Disabling compacts and flushes for region at 1675900418118Disabling writes for close at 1675900418118Writing region close event to WAL at 1675900418118Closed at 1675900418118 2023-02-08 23:53:38,123 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:53:38,139 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675900418126"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900418126"}]},"ts":"1675900418126"} 2023-02-08 23:53:38,163 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:53:38,165 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:53:38,169 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900418166"}]},"ts":"1675900418166"} 2023-02-08 23:53:38,173 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-08 23:53:38,182 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=31ce00ef22047cfd9f24d7e09ef0a684, ASSIGN}] 2023-02-08 23:53:38,185 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=31ce00ef22047cfd9f24d7e09ef0a684, ASSIGN 2023-02-08 23:53:38,186 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=31ce00ef22047cfd9f24d7e09ef0a684, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:53:38,337 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=31ce00ef22047cfd9f24d7e09ef0a684, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:38,337 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675900418337"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900418337"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900418337"}]},"ts":"1675900418337"} 2023-02-08 23:53:38,343 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 31ce00ef22047cfd9f24d7e09ef0a684, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:53:38,504 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:53:38,504 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 31ce00ef22047cfd9f24d7e09ef0a684, NAME => 'hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:53:38,505 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:53:38,505 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:38,506 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:53:38,506 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:53:38,507 INFO [StoreOpener-31ce00ef22047cfd9f24d7e09ef0a684-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:53:38,510 DEBUG [StoreOpener-31ce00ef22047cfd9f24d7e09ef0a684-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684/info 2023-02-08 23:53:38,510 DEBUG [StoreOpener-31ce00ef22047cfd9f24d7e09ef0a684-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684/info 2023-02-08 23:53:38,510 INFO [StoreOpener-31ce00ef22047cfd9f24d7e09ef0a684-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 31ce00ef22047cfd9f24d7e09ef0a684 columnFamilyName info 2023-02-08 23:53:38,511 INFO [StoreOpener-31ce00ef22047cfd9f24d7e09ef0a684-1] regionserver.HStore(310): Store=31ce00ef22047cfd9f24d7e09ef0a684/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:38,513 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:53:38,514 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:53:38,519 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:53:38,523 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:53:38,524 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 31ce00ef22047cfd9f24d7e09ef0a684; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73677863, jitterRate=0.09788571298122406}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:53:38,524 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 31ce00ef22047cfd9f24d7e09ef0a684: Running coprocessor pre-open hook at 1675900418506Writing region info on filesystem at 1675900418506Initializing all the Stores at 1675900418507 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675900418507Cleaning up temporary data from old regions at 1675900418514 (+7 ms)Cleaning up detritus from prior splits at 1675900418515 (+1 ms)Running coprocessor post-open hooks at 1675900418524 (+9 ms)Region opened successfully at 1675900418524 2023-02-08 23:53:38,526 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684., pid=6, masterSystemTime=1675900418497 2023-02-08 23:53:38,530 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:53:38,530 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:53:38,532 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=31ce00ef22047cfd9f24d7e09ef0a684, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:38,533 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675900418531"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900418531"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900418531"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900418531"}]},"ts":"1675900418531"} 2023-02-08 23:53:38,540 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-08 23:53:38,540 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 31ce00ef22047cfd9f24d7e09ef0a684, server=jenkins-hbase4.apache.org,42703,1675900414768 in 193 msec 2023-02-08 23:53:38,543 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-08 23:53:38,544 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=31ce00ef22047cfd9f24d7e09ef0a684, ASSIGN in 358 msec 2023-02-08 23:53:38,545 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:53:38,545 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900418545"}]},"ts":"1675900418545"} 2023-02-08 23:53:38,548 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-08 23:53:38,552 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:53:38,554 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-08 23:53:38,554 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.3100 sec 2023-02-08 23:53:38,555 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-08 23:53:38,555 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:53:38,594 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-08 23:53:38,607 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 23:53:38,613 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 30 msec 2023-02-08 23:53:38,617 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-08 23:53:38,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 23:53:38,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-02-08 23:53:38,644 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-08 23:53:38,646 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-08 23:53:38,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 3.809sec 2023-02-08 23:53:38,647 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-08 23:53:38,649 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-08 23:53:38,649 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-08 23:53:38,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39489,1675900413627-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-08 23:53:38,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39489,1675900413627-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-08 23:53:38,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-08 23:53:38,707 DEBUG [Listener at localhost/41521] zookeeper.ReadOnlyZKClient(139): Connect 0x0672b468 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:53:38,712 DEBUG [Listener at localhost/41521] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@561bf69b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:53:38,725 DEBUG [hconnection-0x78f916d2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:53:38,736 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36958, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:53:38,747 INFO [Listener at localhost/41521] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:53:38,748 DEBUG [Listener at localhost/41521] zookeeper.ReadOnlyZKClient(139): Connect 0x33fc1e18 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:53:38,753 DEBUG [Listener at localhost/41521] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b586bb5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:53:38,753 INFO [Listener at localhost/41521] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:53:38,774 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=328, ProcessCount=168, AvailableMemoryMB=4205 2023-02-08 23:53:38,777 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:38,777 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:38,784 DEBUG [Listener at localhost/41521] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:53:38,787 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45222, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:53:38,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 23:53:38,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:38,802 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:53:38,804 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:53:38,807 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:38,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-08 23:53:38,808 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c empty. 2023-02-08 23:53:38,810 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:38,810 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-08 23:53:38,815 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:38,815 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:38,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 23:53:38,922 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:38,922 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:38,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 23:53:39,124 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:39,124 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:39,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 23:53:39,227 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-08 23:53:39,229 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 893b8071d052c13d0194a6716bc74a8c, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:53:39,241 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:39,241 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing 893b8071d052c13d0194a6716bc74a8c, disabling compactions & flushes 2023-02-08 23:53:39,241 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:39,241 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:39,241 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. after waiting 0 ms 2023-02-08 23:53:39,241 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:39,241 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:39,241 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for 893b8071d052c13d0194a6716bc74a8c: Waiting for close lock at 1675900419241Disabling compacts and flushes for region at 1675900419241Disabling writes for close at 1675900419241Writing region close event to WAL at 1675900419241Closed at 1675900419241 2023-02-08 23:53:39,245 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:53:39,247 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675900419247"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900419247"}]},"ts":"1675900419247"} 2023-02-08 23:53:39,249 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:53:39,251 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:53:39,251 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900419251"}]},"ts":"1675900419251"} 2023-02-08 23:53:39,253 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-08 23:53:39,257 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=893b8071d052c13d0194a6716bc74a8c, ASSIGN}] 2023-02-08 23:53:39,258 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=893b8071d052c13d0194a6716bc74a8c, ASSIGN 2023-02-08 23:53:39,260 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=893b8071d052c13d0194a6716bc74a8c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:53:39,411 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=893b8071d052c13d0194a6716bc74a8c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:39,411 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675900419411"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900419411"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900419411"}]},"ts":"1675900419411"} 2023-02-08 23:53:39,415 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 893b8071d052c13d0194a6716bc74a8c, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:53:39,426 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:39,427 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:39,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 23:53:39,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:39,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 893b8071d052c13d0194a6716bc74a8c, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:53:39,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:39,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:39,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:39,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:39,577 INFO [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:39,579 DEBUG [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:39,579 DEBUG [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:39,582 INFO [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 893b8071d052c13d0194a6716bc74a8c columnFamilyName family1 2023-02-08 23:53:39,585 INFO [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] regionserver.HStore(310): Store=893b8071d052c13d0194a6716bc74a8c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:39,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:39,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:39,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:39,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:53:39,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 893b8071d052c13d0194a6716bc74a8c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65783620, jitterRate=-0.019747674465179443}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:53:39,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 893b8071d052c13d0194a6716bc74a8c: Running coprocessor pre-open hook at 1675900419575Writing region info on filesystem at 1675900419575Initializing all the Stores at 1675900419576 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675900419576Cleaning up temporary data from old regions at 1675900419589 (+13 ms)Cleaning up detritus from prior splits at 1675900419589Running coprocessor post-open hooks at 1675900419597 (+8 ms)Region opened successfully at 1675900419597 2023-02-08 23:53:39,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c., pid=11, masterSystemTime=1675900419568 2023-02-08 23:53:39,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:39,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:39,602 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=893b8071d052c13d0194a6716bc74a8c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:39,602 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675900419602"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900419602"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900419602"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900419602"}]},"ts":"1675900419602"} 2023-02-08 23:53:39,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-08 23:53:39,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 893b8071d052c13d0194a6716bc74a8c, server=jenkins-hbase4.apache.org,42703,1675900414768 in 190 msec 2023-02-08 23:53:39,612 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-08 23:53:39,612 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=893b8071d052c13d0194a6716bc74a8c, ASSIGN in 352 msec 2023-02-08 23:53:39,613 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:53:39,614 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900419613"}]},"ts":"1675900419613"} 2023-02-08 23:53:39,616 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-08 23:53:39,619 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:53:39,621 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 820 msec 2023-02-08 23:53:39,928 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:39,929 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:39,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 23:53:39,931 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-08 23:53:39,948 DEBUG [hconnection-0x7aa6bd5c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:53:39,951 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36972, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:53:40,102 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:40,102 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:40,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:40,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,120 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:40,120 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:40,121 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:40,121 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:40,121 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,122 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,124 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:40,124 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:40,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:40,125 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,125 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:40,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:40,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,127 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,144 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:40,144 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:40,145 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:40,146 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:40,147 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:40,147 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:40,148 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:40,149 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 23:53:40,713 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113048a23174422b4bbabbb2ae83b5a85f80 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113048a23174422b4bbabbb2ae83b5a85f80 2023-02-08 23:53:40,729 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/ee5a1a67fa9f4ed6b5ac83339adfcc24 2023-02-08 23:53:40,739 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/ee5a1a67fa9f4ed6b5ac83339adfcc24 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/ee5a1a67fa9f4ed6b5ac83339adfcc24 2023-02-08 23:53:40,749 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/ee5a1a67fa9f4ed6b5ac83339adfcc24, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-08 23:53:40,751 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 602ms, sequenceid=5, compaction requested=false 2023-02-08 23:53:40,753 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:40,755 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:40,755 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:40,755 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:40,756 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:40,756 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,756 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:40,756 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:40,758 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,759 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:40,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:40,759 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,759 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:40,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:40,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:40,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:40,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:40,761 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:40,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5aacd0aa[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:40,762 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,764 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,764 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:40,764 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:40,764 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,764 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:40,764 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:40,766 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,767 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:40,767 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:40,767 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:40,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:40,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:40,771 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:40,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:40,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:40,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:40,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,772 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:40,775 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,776 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:40,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,776 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:40,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:40,777 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:40,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,778 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:40,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,785 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,785 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,785 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:40,785 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:40,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:40,785 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:40,786 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,786 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,786 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,786 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,786 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:40,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:40,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:40,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:40,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:40,788 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:40,788 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:40,888 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:40,889 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:40,889 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:40,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:40,903 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:40,903 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:40,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:40,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:40,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,907 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:40,907 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:40,907 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:40,907 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:40,908 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,908 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,910 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:40,910 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:40,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:40,910 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:40,910 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:40,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,912 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,912 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 23:53:40,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:40,912 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:40,912 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:40,913 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:40,913 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:40,913 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:40,913 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:40,913 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:40,913 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 23:53:41,350 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128295575d82f8f4e168cdbfdf0fb91e028 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128295575d82f8f4e168cdbfdf0fb91e028 2023-02-08 23:53:41,763 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/5a80be19dc9048c2b0057460e4cb2dd3 2023-02-08 23:53:41,772 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/5a80be19dc9048c2b0057460e4cb2dd3 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/5a80be19dc9048c2b0057460e4cb2dd3 2023-02-08 23:53:41,781 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/5a80be19dc9048c2b0057460e4cb2dd3, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 23:53:41,782 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 869ms, sequenceid=9, compaction requested=false 2023-02-08 23:53:41,783 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:41,783 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:41,783 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:41,784 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:41,784 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,784 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:41,784 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:41,786 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,786 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:41,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:41,786 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,786 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:41,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:41,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:41,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:41,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:41,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50b6a247[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:41,789 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:41,789 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,792 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,792 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:41,792 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:41,792 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:41,792 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,792 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:41,794 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:41,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:41,794 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:41,794 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:41,794 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:41,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:41,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:41,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:41,798 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:41,799 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:41,799 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:41,799 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,799 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:41,803 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,803 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,803 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:41,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:41,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:41,804 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,806 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:41,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:41,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:41,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:41,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:41,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:41,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:41,812 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:41,811 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:41,812 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:41,812 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 23:53:41,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:41,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:41,912 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:41,913 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:41,913 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:41,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:41,918 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-08 23:53:41,920 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-08 23:53:41,924 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:41,924 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:41,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:41,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:41,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,928 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:41,929 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:41,929 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:41,929 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:41,933 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,933 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,984 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,984 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:41,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:41,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:41,985 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:41,985 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:41,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:41,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,986 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,986 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 23:53:41,986 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:41,986 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:41,986 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:41,987 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:41,987 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:41,987 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:41,987 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:41,987 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:41,987 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:42,020 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201ab76f56779134b288562c30022074605 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201ab76f56779134b288562c30022074605 2023-02-08 23:53:42,039 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/92d200d7dc1747328e97893c86a99825 2023-02-08 23:53:42,048 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/92d200d7dc1747328e97893c86a99825 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/92d200d7dc1747328e97893c86a99825 2023-02-08 23:53:42,057 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/92d200d7dc1747328e97893c86a99825, entries=1, sequenceid=13, filesize=5.0 K 2023-02-08 23:53:42,058 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 71ms, sequenceid=13, compaction requested=true 2023-02-08 23:53:42,058 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:42,059 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,060 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:42,060 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:42,138 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,138 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:42,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:42,139 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,139 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:42,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:42,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:42,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:42,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:42,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fe5064d[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:42,142 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:42,142 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,192 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,192 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:42,192 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:42,192 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:42,192 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,192 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:42,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:42,246 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:42,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:42,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:42,247 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:42,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:42,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:42,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:42,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:42,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:42,252 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:42,253 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:42,253 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:42,253 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,253 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:42,322 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,322 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,323 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:42,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:42,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:42,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:42,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:42,324 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:42,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:42,328 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:42,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,376 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,377 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:42,377 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:42,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:42,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:42,377 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:42,377 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:42,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:42,377 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,378 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,378 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,378 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 23:53:42,378 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,378 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:42,378 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:42,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:42,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:42,478 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:42,479 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:42,479 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:42,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:42,489 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:42,489 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:42,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:42,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:42,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,494 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:42,495 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:42,495 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:42,495 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:42,496 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,496 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,507 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,507 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:42,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:42,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:42,508 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,508 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:42,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:42,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 23:53:42,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,510 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:42,510 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:42,510 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:42,511 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:42,511 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:42,511 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:42,511 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:42,511 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:42,580 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:53:42,949 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120534fd5b23ff05437fa46243e7ed74b70b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120534fd5b23ff05437fa46243e7ed74b70b 2023-02-08 23:53:42,961 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/d7e71d8eba78414d8e084d77eb9a80c5 2023-02-08 23:53:42,970 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/d7e71d8eba78414d8e084d77eb9a80c5 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d7e71d8eba78414d8e084d77eb9a80c5 2023-02-08 23:53:42,980 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d7e71d8eba78414d8e084d77eb9a80c5, entries=1, sequenceid=17, filesize=5.0 K 2023-02-08 23:53:42,981 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 470ms, sequenceid=17, compaction requested=true 2023-02-08 23:53:42,981 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:42,981 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:42,982 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:42,984 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,984 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:42,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:42,985 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:42,985 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:42,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:42,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:42,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:42,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c28334c[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:42,987 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:42,987 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,988 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,988 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:42,988 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:42,988 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:42,988 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,988 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:42,990 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:42,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:42,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:42,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:42,994 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:42,994 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:42,994 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:42,994 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,994 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:42,997 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,997 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,997 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:42,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:42,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:42,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:42,998 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:42,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:42,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:42,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:42,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:42,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,000 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:43,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,004 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,004 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,004 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:43,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:43,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:43,004 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:43,004 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,005 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,005 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,005 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,005 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,005 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:43,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 23:53:43,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:43,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:43,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:43,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:43,006 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:43,006 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:43,106 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:43,106 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:43,106 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:43,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:43,115 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:43,116 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:43,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:43,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:43,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,119 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:43,119 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:43,120 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:43,120 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:43,120 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,121 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,122 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,122 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:43,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:43,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:43,122 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,122 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:43,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:43,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,123 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,124 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 23:53:43,124 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,124 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:43,124 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:43,124 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:43,125 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:43,125 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:43,125 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:43,125 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:43,125 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:43,580 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122821ba7162bb4c442fa73fdd497aa47e12 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122821ba7162bb4c442fa73fdd497aa47e12 2023-02-08 23:53:43,590 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/1185002ca6b34ebebe54dbef5d8a4863 2023-02-08 23:53:43,598 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/1185002ca6b34ebebe54dbef5d8a4863 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/1185002ca6b34ebebe54dbef5d8a4863 2023-02-08 23:53:43,604 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/1185002ca6b34ebebe54dbef5d8a4863, entries=1, sequenceid=21, filesize=5.0 K 2023-02-08 23:53:43,606 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 481ms, sequenceid=21, compaction requested=true 2023-02-08 23:53:43,606 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:43,606 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:43,606 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:43,606 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:43,606 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,607 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:43,607 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:43,612 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,612 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:43,613 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,613 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:43,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:43,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:43,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40bd95e3[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:43,615 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:43,615 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,616 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:43,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:43,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:43,616 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,616 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:43,618 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:43,618 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,618 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:43,618 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:43,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:43,621 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:43,621 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:43,621 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:43,621 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,621 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:43,623 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,623 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,623 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:43,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:43,624 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:43,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,625 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:43,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,629 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,629 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:43,629 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,629 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:43,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:43,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:43,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 23:53:43,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:43,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:43,629 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:43,629 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,630 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,630 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,630 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,630 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,630 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:43,630 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:43,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:43,731 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:43,731 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:43,731 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:43,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:43,739 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:43,739 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:43,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:43,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:43,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,742 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:43,742 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:43,743 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:43,743 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:43,743 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,743 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,744 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,744 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:43,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:43,745 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:43,745 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 23:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:43,746 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:43,746 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:43,747 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:43,747 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:43,747 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:43,747 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:43,747 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:43,747 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:43,792 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 23:53:43,792 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-08 23:53:44,168 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123176da141567bb45da84196b7dc5139d27 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123176da141567bb45da84196b7dc5139d27 2023-02-08 23:53:44,178 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/771375ea4c3340eebd7aacfafaa7b629 2023-02-08 23:53:44,186 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/771375ea4c3340eebd7aacfafaa7b629 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/771375ea4c3340eebd7aacfafaa7b629 2023-02-08 23:53:44,193 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/771375ea4c3340eebd7aacfafaa7b629, entries=1, sequenceid=25, filesize=5.0 K 2023-02-08 23:53:44,194 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 447ms, sequenceid=25, compaction requested=true 2023-02-08 23:53:44,194 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:44,195 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:44,195 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,196 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:44,196 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:44,197 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,197 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:44,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:44,198 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,198 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25fcb329[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:44,200 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:44,200 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,201 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,201 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:44,201 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:44,201 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:44,201 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,201 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:44,204 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,204 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:44,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:44,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:44,204 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:44,204 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:44,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:44,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:44,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:44,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:44,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:44,207 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:44,207 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:44,208 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:44,208 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,208 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:44,209 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,209 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:44,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:44,209 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:44,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:44,210 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:44,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:44,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:44,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,212 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:44,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,215 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,215 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:44,215 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:44,215 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 23:53:44,215 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:44,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:44,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:44,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:44,215 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:44,215 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,216 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,216 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,216 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,216 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,216 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:44,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:44,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:44,216 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:44,316 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:44,317 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:44,317 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:44,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:44,324 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:44,324 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:44,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:44,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:44,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,327 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:44,328 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:44,328 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:44,328 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:44,329 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,329 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,330 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,330 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:44,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:44,330 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:44,330 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,332 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 23:53:44,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:44,332 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:44,332 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:44,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:44,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:44,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:44,333 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:44,333 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:44,333 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:44,758 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601016bc83351047b4cd58aaaef7ca95b37f6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601016bc83351047b4cd58aaaef7ca95b37f6 2023-02-08 23:53:45,169 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/df4f83c3420844a984f94d65ba24c942 2023-02-08 23:53:45,177 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/df4f83c3420844a984f94d65ba24c942 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/df4f83c3420844a984f94d65ba24c942 2023-02-08 23:53:45,185 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/df4f83c3420844a984f94d65ba24c942, entries=1, sequenceid=29, filesize=5.0 K 2023-02-08 23:53:45,186 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 853ms, sequenceid=29, compaction requested=true 2023-02-08 23:53:45,187 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:45,187 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:45,187 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:45,192 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,192 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:45,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:45,192 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,193 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:45,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:45,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:45,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:45,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:45,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@222c0eb5[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:45,194 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:45,195 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,196 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,196 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:45,196 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:45,196 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,196 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:45,196 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:45,197 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:45,197 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:45,197 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,197 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:45,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:45,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:45,201 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:45,201 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:45,201 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:45,201 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,201 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:45,203 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,203 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:45,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,203 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:45,203 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:45,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,205 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:45,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,211 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,211 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:45,211 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:45,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:45,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:45,211 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:45,211 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,212 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,211 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:45,212 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,212 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:45,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:45,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:45,212 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,212 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 23:53:45,212 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,212 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:45,313 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:45,313 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:45,313 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:45,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:45,320 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:45,320 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:45,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:45,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:45,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,323 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:45,323 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:45,324 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:45,324 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:45,324 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,325 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,326 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:45,326 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:45,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:45,326 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:45,326 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:45,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:45,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 23:53:45,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:45,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:45,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:45,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:45,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:45,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:45,328 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:45,328 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:45,329 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:45,751 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103bbd84b8110bc41cbb36dd57a26d98529 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103bbd84b8110bc41cbb36dd57a26d98529 2023-02-08 23:53:46,188 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/693e9d115f294002aeebdd4395eda3ce 2023-02-08 23:53:46,196 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/693e9d115f294002aeebdd4395eda3ce as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/693e9d115f294002aeebdd4395eda3ce 2023-02-08 23:53:46,204 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/693e9d115f294002aeebdd4395eda3ce, entries=1, sequenceid=33, filesize=5.0 K 2023-02-08 23:53:46,206 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 877ms, sequenceid=33, compaction requested=true 2023-02-08 23:53:46,206 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:46,206 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:46,206 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:46,206 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:46,206 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:46,207 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:46,208 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,208 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:46,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:46,209 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,209 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:46,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:46,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fa3ebef[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:46,210 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:46,210 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,212 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,212 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:46,212 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:46,212 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:46,212 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,212 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:46,214 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:46,214 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,214 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:46,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:46,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:46,214 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:46,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:46,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:46,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:46,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:46,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:46,217 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:46,217 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:46,217 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:46,217 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,217 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:46,219 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,219 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:46,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,219 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:46,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:46,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:46,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:46,220 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:46,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:46,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,221 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:46,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,224 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,224 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:46,224 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,224 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:46,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:46,224 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:46,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:46,224 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,224 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:46,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:46,224 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,225 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,225 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:46,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:46,225 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:46,225 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:46,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 23:53:46,325 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:46,326 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:46,326 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:46,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:46,333 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:46,333 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:46,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:46,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:46,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,336 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:46,336 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:46,337 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:46,337 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:46,337 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,337 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,339 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:46,339 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:46,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:46,340 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,340 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:46,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,341 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,341 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 23:53:46,341 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:46,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:46,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:46,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:46,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:46,342 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:46,342 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:46,343 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:46,763 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907feb1ca6066b04b1192c156590bed302d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907feb1ca6066b04b1192c156590bed302d 2023-02-08 23:53:46,772 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/0b535170e26b4915887e02860118c9bf 2023-02-08 23:53:46,778 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/0b535170e26b4915887e02860118c9bf as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/0b535170e26b4915887e02860118c9bf 2023-02-08 23:53:46,785 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/0b535170e26b4915887e02860118c9bf, entries=1, sequenceid=37, filesize=5.0 K 2023-02-08 23:53:46,786 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 443ms, sequenceid=37, compaction requested=true 2023-02-08 23:53:46,786 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:46,786 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:46,786 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:46,787 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:46,787 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,787 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:46,787 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:46,792 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:46,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:46,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:46,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:46,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:46,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:46,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@339d7775[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:46,794 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:46,794 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,795 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,796 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:46,796 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:46,796 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:46,796 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,796 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:46,797 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:46,797 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,797 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:46,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:46,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:46,797 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:46,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:46,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:46,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:46,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:46,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:46,800 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:46,800 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:46,800 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:46,800 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,800 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:46,802 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,802 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,802 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:46,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:46,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:46,803 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,804 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:46,811 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:46,812 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:46,812 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:46,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:46,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:46,812 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:46,812 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,812 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,812 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,812 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:46,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:46,812 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:46,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:46,913 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:46,913 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:46,913 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:46,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:46,920 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:46,920 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:46,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:46,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:46,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,923 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:46,923 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:46,924 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 23:53:46,924 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 23:53:46,924 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,924 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,925 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,925 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:46,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:46,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:46,926 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:46,926 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:46,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:46,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,927 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,927 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 23:53:46,927 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 23:53:46,927 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:46,928 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 23:53:46,928 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:46,928 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:46,929 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:46,929 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. started... 2023-02-08 23:53:46,929 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 893b8071d052c13d0194a6716bc74a8c 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:47,349 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112080b9db6e70f244b8ad0552d3cf4d7998 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112080b9db6e70f244b8ad0552d3cf4d7998 2023-02-08 23:53:47,758 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/d55d7d3296f5450db7c0a31dee582c7c 2023-02-08 23:53:47,766 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/d55d7d3296f5450db7c0a31dee582c7c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d55d7d3296f5450db7c0a31dee582c7c 2023-02-08 23:53:47,773 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d55d7d3296f5450db7c0a31dee582c7c, entries=1, sequenceid=41, filesize=5.0 K 2023-02-08 23:53:47,774 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 893b8071d052c13d0194a6716bc74a8c in 845ms, sequenceid=41, compaction requested=true 2023-02-08 23:53:47,775 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:47,775 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:47,775 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:47,775 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:47,775 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,775 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 23:53:47,775 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 23:53:47,777 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:47,778 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:47,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:47,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:47,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 23:53:47,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7338efd4[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:47,779 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 23:53:47,779 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,780 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,780 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 23:53:47,780 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 23:53:47,780 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:47,780 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,780 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:47,790 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,790 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 23:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:47,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:47,790 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:47,790 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 23:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:47,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 23:53:47,794 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 23:53:47,794 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:47,794 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:47,794 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,794 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:47,796 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,796 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:47,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,796 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:47,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:47,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:47,796 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,798 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,800 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,800 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:47,800 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:47,800 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:47,800 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:47,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 23:53:47,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:47,800 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:47,800 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:47,801 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:47,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:47,801 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,801 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,801 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:47,801 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:47,901 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:47,901 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:47,902 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:47,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:47,908 DEBUG [Listener at localhost/41521] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 23:53:47,910 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57756, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 23:53:47,912 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:47,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 23:53:47,928 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 23:53:47,930 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901027930 2023-02-08 23:53:47,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:53:47,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:53:47,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-08 23:53:47,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:47,936 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x3fe8aafd to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:53:47,941 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1284f2b0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:53:47,942 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:48,812 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103bc2f0a1ff58d4cbcbe996861d3885ed5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103bc2f0a1ff58d4cbcbe996861d3885ed5 2023-02-08 23:53:48,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:48,832 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:53:48,833 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50224, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:53:48,844 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:48,844 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:48,845 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:53:48,847 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57760, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:53:48,880 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dd5bc4c2f1e747da9ba010a355fa45ad first=Optional[r3] last=Optional[r6] 2023-02-08 23:53:48,892 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dd5bc4c2f1e747da9ba010a355fa45ad}] 2023-02-08 23:53:48,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dd5bc4c2f1e747da9ba010a355fa45ad for inclusion in 893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:48,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 23:53:48,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:53:48,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:48,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dd5bc4c2f1e747da9ba010a355fa45ad to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__i418mn5bjjd69gtqerra6uno2v36gvlgctrcpuocqgarct06vnhoi37q876aj4c0/family1/dd5bc4c2f1e747da9ba010a355fa45ad 2023-02-08 23:53:48,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__i418mn5bjjd69gtqerra6uno2v36gvlgctrcpuocqgarct06vnhoi37q876aj4c0/family1/dd5bc4c2f1e747da9ba010a355fa45ad as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d2b3a7ce7be3444abca6ede256ec7dc7_SeqId_44_ 2023-02-08 23:53:48,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__i418mn5bjjd69gtqerra6uno2v36gvlgctrcpuocqgarct06vnhoi37q876aj4c0/family1/dd5bc4c2f1e747da9ba010a355fa45ad into 893b8071d052c13d0194a6716bc74a8c/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d2b3a7ce7be3444abca6ede256ec7dc7_SeqId_44_ - updating store file list. 2023-02-08 23:53:48,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d2b3a7ce7be3444abca6ede256ec7dc7_SeqId_44_ into 893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:48,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__i418mn5bjjd69gtqerra6uno2v36gvlgctrcpuocqgarct06vnhoi37q876aj4c0/family1/dd5bc4c2f1e747da9ba010a355fa45ad into 893b8071d052c13d0194a6716bc74a8c/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d2b3a7ce7be3444abca6ede256ec7dc7_SeqId_44_) 2023-02-08 23:53:48,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__i418mn5bjjd69gtqerra6uno2v36gvlgctrcpuocqgarct06vnhoi37q876aj4c0/family1/dd5bc4c2f1e747da9ba010a355fa45ad 2023-02-08 23:53:48,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 23:53:48,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 23:53:48,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 893b8071d052c13d0194a6716bc74a8c/family1 is initiating minor compaction 2023-02-08 23:53:48,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@70c7234b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:53:48,941 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 893b8071d052c13d0194a6716bc74a8c/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:48,942 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/5a80be19dc9048c2b0057460e4cb2dd3, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/92d200d7dc1747328e97893c86a99825, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d7e71d8eba78414d8e084d77eb9a80c5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/1185002ca6b34ebebe54dbef5d8a4863, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/771375ea4c3340eebd7aacfafaa7b629, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/df4f83c3420844a984f94d65ba24c942, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/693e9d115f294002aeebdd4395eda3ce, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/0b535170e26b4915887e02860118c9bf, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d55d7d3296f5450db7c0a31dee582c7c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d2b3a7ce7be3444abca6ede256ec7dc7_SeqId_44_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp, totalSize=50.2 K 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 5a80be19dc9048c2b0057460e4cb2dd3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 92d200d7dc1747328e97893c86a99825, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting d7e71d8eba78414d8e084d77eb9a80c5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 1185002ca6b34ebebe54dbef5d8a4863, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 771375ea4c3340eebd7aacfafaa7b629, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting df4f83c3420844a984f94d65ba24c942, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 693e9d115f294002aeebdd4395eda3ce, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 0b535170e26b4915887e02860118c9bf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting d55d7d3296f5450db7c0a31dee582c7c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 23:53:48,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting d2b3a7ce7be3444abca6ede256ec7dc7_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 23:53:48,956 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:48,958 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:53:48,963 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122821ba7162bb4c442fa73fdd497aa47e12 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122821ba7162bb4c442fa73fdd497aa47e12 2023-02-08 23:53:48,965 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123176da141567bb45da84196b7dc5139d27 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123176da141567bb45da84196b7dc5139d27 2023-02-08 23:53:48,967 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601016bc83351047b4cd58aaaef7ca95b37f6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601016bc83351047b4cd58aaaef7ca95b37f6 2023-02-08 23:53:48,969 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103bbd84b8110bc41cbb36dd57a26d98529 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103bbd84b8110bc41cbb36dd57a26d98529 2023-02-08 23:53:48,969 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-08 23:53:48,969 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 23:53:48,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:48,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:48,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:48,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:48,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:48,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:48,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:48,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:48,999 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 893b8071d052c13d0194a6716bc74a8c#family1#compaction#10 average throughput is 0.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:53:49,827 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/a899790904d44096815eb713fd093453 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/a899790904d44096815eb713fd093453 2023-02-08 23:53:49,845 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 893b8071d052c13d0194a6716bc74a8c/family1 of 893b8071d052c13d0194a6716bc74a8c into a899790904d44096815eb713fd093453(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:53:49,845 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:49,847 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1675900428940; duration=0sec 2023-02-08 23:53:49,848 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:53:49,859 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d51269155c2946d8a769f1638482a093 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205d51269155c2946d8a769f1638482a093 2023-02-08 23:53:49,867 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:49,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/5a80be19dc9048c2b0057460e4cb2dd3, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/92d200d7dc1747328e97893c86a99825, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d7e71d8eba78414d8e084d77eb9a80c5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/1185002ca6b34ebebe54dbef5d8a4863, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/771375ea4c3340eebd7aacfafaa7b629, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/df4f83c3420844a984f94d65ba24c942, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/693e9d115f294002aeebdd4395eda3ce, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/0b535170e26b4915887e02860118c9bf, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d55d7d3296f5450db7c0a31dee582c7c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d2b3a7ce7be3444abca6ede256ec7dc7_SeqId_44_] to archive 2023-02-08 23:53:49,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:53:49,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/5a80be19dc9048c2b0057460e4cb2dd3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/5a80be19dc9048c2b0057460e4cb2dd3 2023-02-08 23:53:49,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/92d200d7dc1747328e97893c86a99825 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/92d200d7dc1747328e97893c86a99825 2023-02-08 23:53:49,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d7e71d8eba78414d8e084d77eb9a80c5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d7e71d8eba78414d8e084d77eb9a80c5 2023-02-08 23:53:49,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/1185002ca6b34ebebe54dbef5d8a4863 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/1185002ca6b34ebebe54dbef5d8a4863 2023-02-08 23:53:49,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/771375ea4c3340eebd7aacfafaa7b629 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/771375ea4c3340eebd7aacfafaa7b629 2023-02-08 23:53:49,887 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/df4f83c3420844a984f94d65ba24c942 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/df4f83c3420844a984f94d65ba24c942 2023-02-08 23:53:49,889 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/693e9d115f294002aeebdd4395eda3ce to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/693e9d115f294002aeebdd4395eda3ce 2023-02-08 23:53:49,891 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/0b535170e26b4915887e02860118c9bf to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/0b535170e26b4915887e02860118c9bf 2023-02-08 23:53:49,893 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d55d7d3296f5450db7c0a31dee582c7c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d55d7d3296f5450db7c0a31dee582c7c 2023-02-08 23:53:49,894 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:49,895 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:49,896 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d2b3a7ce7be3444abca6ede256ec7dc7_SeqId_44_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d2b3a7ce7be3444abca6ede256ec7dc7_SeqId_44_ 2023-02-08 23:53:49,909 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cbd7769541af4628b6410a30305f6d64 first=Optional[r00] last=Optional[r2] 2023-02-08 23:53:49,910 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cbd7769541af4628b6410a30305f6d64}] 2023-02-08 23:53:49,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cbd7769541af4628b6410a30305f6d64 for inclusion in 893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:49,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-08 23:53:49,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:53:49,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:49,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/cbd7769541af4628b6410a30305f6d64 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__aug9nlgk7b34l3c9f290cl158p5l10qpr59optovm1tt5h0mnvee6drps5p6qalm/family1/cbd7769541af4628b6410a30305f6d64 2023-02-08 23:53:49,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__aug9nlgk7b34l3c9f290cl158p5l10qpr59optovm1tt5h0mnvee6drps5p6qalm/family1/cbd7769541af4628b6410a30305f6d64 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/7c8a60eb7bf241a7951724fe73a00056_SeqId_47_ 2023-02-08 23:53:49,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__aug9nlgk7b34l3c9f290cl158p5l10qpr59optovm1tt5h0mnvee6drps5p6qalm/family1/cbd7769541af4628b6410a30305f6d64 into 893b8071d052c13d0194a6716bc74a8c/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/7c8a60eb7bf241a7951724fe73a00056_SeqId_47_ - updating store file list. 2023-02-08 23:53:49,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/7c8a60eb7bf241a7951724fe73a00056_SeqId_47_ into 893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:49,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__aug9nlgk7b34l3c9f290cl158p5l10qpr59optovm1tt5h0mnvee6drps5p6qalm/family1/cbd7769541af4628b6410a30305f6d64 into 893b8071d052c13d0194a6716bc74a8c/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/7c8a60eb7bf241a7951724fe73a00056_SeqId_47_) 2023-02-08 23:53:49,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__aug9nlgk7b34l3c9f290cl158p5l10qpr59optovm1tt5h0mnvee6drps5p6qalm/family1/cbd7769541af4628b6410a30305f6d64 2023-02-08 23:53:49,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 23:53:49,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 23:53:49,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 893b8071d052c13d0194a6716bc74a8c/family1 is initiating minor compaction (all files) 2023-02-08 23:53:49,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@24da50a3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:53:49,935 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 893b8071d052c13d0194a6716bc74a8c/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:49,935 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/ee5a1a67fa9f4ed6b5ac83339adfcc24, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/a899790904d44096815eb713fd093453, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/7c8a60eb7bf241a7951724fe73a00056_SeqId_47_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp, totalSize=358.6 K 2023-02-08 23:53:49,936 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting ee5a1a67fa9f4ed6b5ac83339adfcc24, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838754 2023-02-08 23:53:49,937 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting a899790904d44096815eb713fd093453, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238754 2023-02-08 23:53:49,937 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 7c8a60eb7bf241a7951724fe73a00056_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878838754 2023-02-08 23:53:49,938 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:49,939 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:53:49,941 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113048a23174422b4bbabbb2ae83b5a85f80 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113048a23174422b4bbabbb2ae83b5a85f80 2023-02-08 23:53:49,943 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201ab76f56779134b288562c30022074605 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201ab76f56779134b288562c30022074605 2023-02-08 23:53:49,944 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120534fd5b23ff05437fa46243e7ed74b70b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120534fd5b23ff05437fa46243e7ed74b70b 2023-02-08 23:53:49,944 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-08 23:53:49,944 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 23:53:49,945 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:49,945 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:49,945 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:53:49,946 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 23:53:49,949 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0290 sec 2023-02-08 23:53:49,978 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 893b8071d052c13d0194a6716bc74a8c#family1#compaction#11 average throughput is 6.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:53:50,108 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:50,108 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:50,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 23:53:50,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:50,534 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-08 23:53:50,536 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-08 23:53:50,536 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-08 23:53:50,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-08 23:53:50,553 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=893b8071d052c13d0194a6716bc74a8c, REOPEN/MOVE}] 2023-02-08 23:53:50,554 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=893b8071d052c13d0194a6716bc74a8c, REOPEN/MOVE 2023-02-08 23:53:50,555 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=893b8071d052c13d0194a6716bc74a8c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:50,555 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675900430555"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900430555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900430555"}]},"ts":"1675900430555"} 2023-02-08 23:53:50,557 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 893b8071d052c13d0194a6716bc74a8c, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:53:50,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:50,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 893b8071d052c13d0194a6716bc74a8c, disabling compactions & flushes 2023-02-08 23:53:50,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:50,819 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/.tmp/family1/a40ff4e32585400390bcb30a1cda1e27 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/a40ff4e32585400390bcb30a1cda1e27 2023-02-08 23:53:50,827 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 893b8071d052c13d0194a6716bc74a8c/family1 of 893b8071d052c13d0194a6716bc74a8c into a40ff4e32585400390bcb30a1cda1e27(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:53:50,827 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:50,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:50,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:50,827 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1675900429935; duration=0sec 2023-02-08 23:53:50,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. after waiting 0 ms 2023-02-08 23:53:50,827 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:53:50,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:50,831 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/ee5a1a67fa9f4ed6b5ac83339adfcc24, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/a899790904d44096815eb713fd093453, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/7c8a60eb7bf241a7951724fe73a00056_SeqId_47_] to archive 2023-02-08 23:53:50,832 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:53:50,835 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/ee5a1a67fa9f4ed6b5ac83339adfcc24 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/ee5a1a67fa9f4ed6b5ac83339adfcc24 2023-02-08 23:53:50,836 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/a899790904d44096815eb713fd093453 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/a899790904d44096815eb713fd093453 2023-02-08 23:53:50,838 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/7c8a60eb7bf241a7951724fe73a00056_SeqId_47_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/7c8a60eb7bf241a7951724fe73a00056_SeqId_47_ 2023-02-08 23:53:50,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-08 23:53:50,846 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:50,846 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 893b8071d052c13d0194a6716bc74a8c: Waiting for close lock at 1675900430716Running coprocessor pre-close hooks at 1675900430716Disabling compacts and flushes for region at 1675900430716Disabling writes for close at 1675900430827 (+111 ms)Writing region close event to WAL at 1675900430841 (+14 ms)Running coprocessor post-close hooks at 1675900430845 (+4 ms)Closed at 1675900430846 (+1 ms) 2023-02-08 23:53:50,846 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 893b8071d052c13d0194a6716bc74a8c to self. 2023-02-08 23:53:50,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:50,848 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=893b8071d052c13d0194a6716bc74a8c, regionState=CLOSED 2023-02-08 23:53:50,849 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675900430848"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900430848"}]},"ts":"1675900430848"} 2023-02-08 23:53:50,853 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-08 23:53:50,853 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 893b8071d052c13d0194a6716bc74a8c, server=jenkins-hbase4.apache.org,42703,1675900414768 in 294 msec 2023-02-08 23:53:50,854 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=893b8071d052c13d0194a6716bc74a8c, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=true 2023-02-08 23:53:51,005 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=893b8071d052c13d0194a6716bc74a8c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:51,005 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675900431004"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900431004"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900431004"}]},"ts":"1675900431004"} 2023-02-08 23:53:51,007 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 893b8071d052c13d0194a6716bc74a8c, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:53:51,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:51,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 893b8071d052c13d0194a6716bc74a8c, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:53:51,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:51,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:51,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:51,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:51,166 INFO [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:51,167 DEBUG [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:51,167 DEBUG [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:51,168 INFO [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 893b8071d052c13d0194a6716bc74a8c columnFamilyName family1 2023-02-08 23:53:51,175 DEBUG [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/a40ff4e32585400390bcb30a1cda1e27 2023-02-08 23:53:51,175 INFO [StoreOpener-893b8071d052c13d0194a6716bc74a8c-1] regionserver.HStore(310): Store=893b8071d052c13d0194a6716bc74a8c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:51,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:51,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:51,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:53:51,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 893b8071d052c13d0194a6716bc74a8c; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70653047, jitterRate=0.05281244218349457}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:53:51,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 893b8071d052c13d0194a6716bc74a8c: Running coprocessor pre-open hook at 1675900431165Writing region info on filesystem at 1675900431165Initializing all the Stores at 1675900431165Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675900431166 (+1 ms)Cleaning up temporary data from old regions at 1675900431178 (+12 ms)Cleaning up detritus from prior splits at 1675900431179 (+1 ms)Running coprocessor post-open hooks at 1675900431182 (+3 ms)Region opened successfully at 1675900431182 2023-02-08 23:53:51,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c., pid=17, masterSystemTime=1675900431160 2023-02-08 23:53:51,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:51,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:53:51,185 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=893b8071d052c13d0194a6716bc74a8c, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:51,186 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675900431185"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900431185"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900431185"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900431185"}]},"ts":"1675900431185"} 2023-02-08 23:53:51,189 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-08 23:53:51,189 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 893b8071d052c13d0194a6716bc74a8c, server=jenkins-hbase4.apache.org,42703,1675900414768 in 180 msec 2023-02-08 23:53:51,191 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-08 23:53:51,191 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=893b8071d052c13d0194a6716bc74a8c, REOPEN/MOVE in 636 msec 2023-02-08 23:53:51,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-08 23:53:51,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 653 msec 2023-02-08 23:53:51,197 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0790 sec 2023-02-08 23:53:51,199 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:51,199 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:51,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-08 23:53:51,201 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:51,201 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:51,206 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:51,206 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:51,207 INFO [Listener at localhost/41521] compactions.TestMobCompactor(278): alter status finished 2023-02-08 23:53:51,208 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 23:53:51,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 23:53:51,210 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 23:53:51,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:53:51,211 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901031211 2023-02-08 23:53:51,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:53:51,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-08 23:53:51,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:51,213 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x621f5eb2 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:53:51,218 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3e7124db, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:53:51,218 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:51,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:51,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:51,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:51,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:52,056 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289a425a2ed0c24681a47595b19bccb5b0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511289a425a2ed0c24681a47595b19bccb5b0 2023-02-08 23:53:52,057 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:52,070 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:53:52,071 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50226, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:53:52,079 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:52,079 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:52,079 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:53:52,081 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57766, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:53:52,094 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ce57ff78b2654b0bb0955dba83615b46 first=Optional[r05] last=Optional[r8] 2023-02-08 23:53:52,095 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=51 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ce57ff78b2654b0bb0955dba83615b46}] 2023-02-08 23:53:52,102 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ce57ff78b2654b0bb0955dba83615b46 for inclusion in 893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:52,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-08 23:53:52,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:53:52,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 893b8071d052c13d0194a6716bc74a8c: 2023-02-08 23:53:52,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ce57ff78b2654b0bb0955dba83615b46 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__4ved0r6ptincajiulb6aap84ufooplh8754jmh7ochuld7rfuogf83v656mbkgkj/family1/ce57ff78b2654b0bb0955dba83615b46 2023-02-08 23:53:52,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__4ved0r6ptincajiulb6aap84ufooplh8754jmh7ochuld7rfuogf83v656mbkgkj/family1/ce57ff78b2654b0bb0955dba83615b46 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d4b39d1fa8184789a97c6de9e6b7f33a_SeqId_53_ 2023-02-08 23:53:52,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__4ved0r6ptincajiulb6aap84ufooplh8754jmh7ochuld7rfuogf83v656mbkgkj/family1/ce57ff78b2654b0bb0955dba83615b46 into 893b8071d052c13d0194a6716bc74a8c/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d4b39d1fa8184789a97c6de9e6b7f33a_SeqId_53_ - updating store file list. 2023-02-08 23:53:52,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d4b39d1fa8184789a97c6de9e6b7f33a_SeqId_53_ into 893b8071d052c13d0194a6716bc74a8c/family1 2023-02-08 23:53:52,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__4ved0r6ptincajiulb6aap84ufooplh8754jmh7ochuld7rfuogf83v656mbkgkj/family1/ce57ff78b2654b0bb0955dba83615b46 into 893b8071d052c13d0194a6716bc74a8c/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/family1/d4b39d1fa8184789a97c6de9e6b7f33a_SeqId_53_) 2023-02-08 23:53:52,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__4ved0r6ptincajiulb6aap84ufooplh8754jmh7ochuld7rfuogf83v656mbkgkj/family1/ce57ff78b2654b0bb0955dba83615b46 2023-02-08 23:53:52,120 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:52,121 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:53:52,122 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112080b9db6e70f244b8ad0552d3cf4d7998 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112080b9db6e70f244b8ad0552d3cf4d7998 2023-02-08 23:53:52,124 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128295575d82f8f4e168cdbfdf0fb91e028 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128295575d82f8f4e168cdbfdf0fb91e028 2023-02-08 23:53:52,124 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-08 23:53:52,124 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 23:53:52,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:52,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:52,125 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 23:53:52,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 23:53:52,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:53:52,126 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 23:53:52,128 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 918 msec 2023-02-08 23:53:52,206 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=249 (was 211) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54774@0x3fe8aafd-SendThread(127.0.0.1:54774) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44598 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44498 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44564 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44566 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44408 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x56ca2f40-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54774@0x621f5eb2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1999313532.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x56ca2f40-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54774@0x621f5eb2-SendThread(127.0.0.1:54774) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/cluster_0361639e-2335-beea-2849-07295ee226f0/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44468 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x30782b0d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:42703-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54774@0x3fe8aafd sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1999313532.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44452 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44390 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x30782b0d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44496 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44646 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44516 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-494294014_15 at /127.0.0.1:44612 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44434 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x30782b0d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44582 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54774@0x621f5eb2-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54774@0x3fe8aafd-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44480 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44642 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=593 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=284 (was 328), ProcessCount=168 (was 168), AvailableMemoryMB=4080 (was 4205) 2023-02-08 23:53:52,218 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=249, OpenFileDescriptor=593, MaxFileDescriptor=60000, SystemLoadAverage=284, ProcessCount=168, AvailableMemoryMB=4080 2023-02-08 23:53:52,219 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:52,219 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:52,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 23:53:52,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:52,224 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:53:52,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-08 23:53:52,225 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:52,225 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:52,225 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:53:52,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 23:53:52,227 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:53:52,227 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0 empty. 2023-02-08 23:53:52,228 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:53:52,228 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-08 23:53:52,326 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:52,326 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:52,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 23:53:52,528 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:52,528 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:52,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 23:53:52,638 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-08 23:53:52,639 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9b83370f2c21546d39bae07b7ba35fa0, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:53:52,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:52,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 9b83370f2c21546d39bae07b7ba35fa0, disabling compactions & flushes 2023-02-08 23:53:52,646 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:52,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:52,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. after waiting 0 ms 2023-02-08 23:53:52,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:52,646 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:52,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 9b83370f2c21546d39bae07b7ba35fa0: Waiting for close lock at 1675900432646Disabling compacts and flushes for region at 1675900432646Disabling writes for close at 1675900432646Writing region close event to WAL at 1675900432646Closed at 1675900432646 2023-02-08 23:53:52,649 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:53:52,650 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900432650"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900432650"}]},"ts":"1675900432650"} 2023-02-08 23:53:52,652 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:53:52,653 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:53:52,653 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900432653"}]},"ts":"1675900432653"} 2023-02-08 23:53:52,654 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-08 23:53:52,699 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=9b83370f2c21546d39bae07b7ba35fa0, ASSIGN}] 2023-02-08 23:53:52,701 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=9b83370f2c21546d39bae07b7ba35fa0, ASSIGN 2023-02-08 23:53:52,702 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=9b83370f2c21546d39bae07b7ba35fa0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:53:52,829 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:52,829 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:52,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 23:53:52,853 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=9b83370f2c21546d39bae07b7ba35fa0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:52,853 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900432853"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900432853"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900432853"}]},"ts":"1675900432853"} 2023-02-08 23:53:52,856 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 9b83370f2c21546d39bae07b7ba35fa0, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:53:53,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:53,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9b83370f2c21546d39bae07b7ba35fa0, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:53:53,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:53:53,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:53:53,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:53:53,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:53:53,014 INFO [StoreOpener-9b83370f2c21546d39bae07b7ba35fa0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:53:53,016 DEBUG [StoreOpener-9b83370f2c21546d39bae07b7ba35fa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1 2023-02-08 23:53:53,016 DEBUG [StoreOpener-9b83370f2c21546d39bae07b7ba35fa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1 2023-02-08 23:53:53,016 INFO [StoreOpener-9b83370f2c21546d39bae07b7ba35fa0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b83370f2c21546d39bae07b7ba35fa0 columnFamilyName family1 2023-02-08 23:53:53,018 INFO [StoreOpener-9b83370f2c21546d39bae07b7ba35fa0-1] regionserver.HStore(310): Store=9b83370f2c21546d39bae07b7ba35fa0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:53:53,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:53:53,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:53:53,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:53:53,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:53:53,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9b83370f2c21546d39bae07b7ba35fa0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68586974, jitterRate=0.02202555537223816}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:53:53,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9b83370f2c21546d39bae07b7ba35fa0: Running coprocessor pre-open hook at 1675900433012Writing region info on filesystem at 1675900433012Initializing all the Stores at 1675900433013 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675900433014 (+1 ms)Cleaning up temporary data from old regions at 1675900433020 (+6 ms)Cleaning up detritus from prior splits at 1675900433021 (+1 ms)Running coprocessor post-open hooks at 1675900433026 (+5 ms)Region opened successfully at 1675900433026 2023-02-08 23:53:53,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0., pid=21, masterSystemTime=1675900433008 2023-02-08 23:53:53,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:53,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:53,029 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=9b83370f2c21546d39bae07b7ba35fa0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:53,030 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900433029"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900433029"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900433029"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900433029"}]},"ts":"1675900433029"} 2023-02-08 23:53:53,033 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-08 23:53:53,033 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 9b83370f2c21546d39bae07b7ba35fa0, server=jenkins-hbase4.apache.org,42703,1675900414768 in 175 msec 2023-02-08 23:53:53,035 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-08 23:53:53,036 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=9b83370f2c21546d39bae07b7ba35fa0, ASSIGN in 334 msec 2023-02-08 23:53:53,036 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:53:53,037 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900433036"}]},"ts":"1675900433036"} 2023-02-08 23:53:53,038 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-08 23:53:53,045 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:53:53,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 824 msec 2023-02-08 23:53:53,331 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:53,331 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:53,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 23:53:53,332 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-08 23:53:53,362 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:53,362 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:53,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:53,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:53,365 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:53,365 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:53,366 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:53:53,366 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:53:53,366 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:53,366 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:53,375 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:53,375 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:53,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:53,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:53,375 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:53,375 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:53,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:53,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:53,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:53,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:53,376 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:53,377 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:53,377 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:53:53,377 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:53,377 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:53:53,378 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:53,378 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:53,378 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:53,378 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:53:53,378 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 23:53:53,804 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130372443120f1146aa8a603c1c81e89ded to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130372443120f1146aa8a603c1c81e89ded 2023-02-08 23:53:53,922 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:54,208 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/50706af476e54c85be9147f41bebb4ae 2023-02-08 23:53:54,214 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/50706af476e54c85be9147f41bebb4ae as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/50706af476e54c85be9147f41bebb4ae 2023-02-08 23:53:54,220 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/50706af476e54c85be9147f41bebb4ae, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-08 23:53:54,221 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 843ms, sequenceid=5, compaction requested=false 2023-02-08 23:53:54,222 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:53:54,222 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:54,222 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:54,222 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:54,222 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,222 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:53:54,222 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:53:54,283 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,283 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:54,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:54,283 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,283 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:54,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:54,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:54,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:54,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:53:54,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31230ea0[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:54,285 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:53:54,285 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,321 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:53:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:53:54,321 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,321 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:54,390 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,390 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:53:54,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:54,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:54,390 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:54,391 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:53:54,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:54,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:54,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:54,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:54,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:53:54,394 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:53:54,394 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:54,394 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:54,394 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,394 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:54,475 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,475 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,475 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:54,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:54,476 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:54,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:54,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:54,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,477 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:54,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,552 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:54,552 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:54,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:54,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:54,552 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:54,552 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,552 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:54,553 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:54,553 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:54,553 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:54,553 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,553 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,553 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,553 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:54,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:54,653 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:54,653 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:54,653 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:54,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:54,659 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:54,659 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:54,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:54,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:54,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,661 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:54,661 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:54,662 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:53:54,662 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:53:54,662 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,663 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,698 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:54,698 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:54,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:54,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,699 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:54,699 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:54,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:54,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,699 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,699 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:53:54,699 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:54,700 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:53:54,700 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:54,700 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:53:54,700 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:54,700 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:54,700 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:54,700 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:53:54,701 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 23:53:55,117 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511287b297ff218ff4eed8e7d9afb8a91872c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511287b297ff218ff4eed8e7d9afb8a91872c 2023-02-08 23:53:55,524 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/d98e59e0efc749d08cea4cf5ba455e80 2023-02-08 23:53:55,531 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/d98e59e0efc749d08cea4cf5ba455e80 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/d98e59e0efc749d08cea4cf5ba455e80 2023-02-08 23:53:55,538 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/d98e59e0efc749d08cea4cf5ba455e80, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 23:53:55,539 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 838ms, sequenceid=9, compaction requested=false 2023-02-08 23:53:55,539 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:53:55,539 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:55,539 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:55,539 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:53:55,540 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:53:55,629 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,629 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:55,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:55,629 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,629 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:55,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:55,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:55,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:55,631 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:53:55,631 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31df77f8[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:55,631 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:53:55,631 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,690 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,691 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:53:55,691 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,691 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:55,691 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:53:55,691 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:55,736 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:53:55,736 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,736 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:55,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:55,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:55,737 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:53:55,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:55,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:55,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:55,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:55,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:53:55,740 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:53:55,740 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:55,740 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:55,740 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,740 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:55,826 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,826 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:55,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:55,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,827 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:55,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:55,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:55,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,828 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,829 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:55,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,946 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:55,946 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:55,947 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:55,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:55,947 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:55,947 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:55,947 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:55,948 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:55,948 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:55,948 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,948 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:55,948 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,948 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:55,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:55,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:56,048 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:56,049 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:56,049 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:56,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:56,054 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:56,054 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:56,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:56,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:56,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,057 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:56,057 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:56,058 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:53:56,058 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:53:56,058 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,058 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,075 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,075 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:56,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:56,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:56,075 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,075 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:56,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:56,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,077 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:53:56,077 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,077 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:53:56,077 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:56,077 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:53:56,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:56,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:56,079 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:56,079 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:53:56,080 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:56,379 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:53:56,501 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201dc3fc3be5ee64ed5b9646f416ea18ca9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201dc3fc3be5ee64ed5b9646f416ea18ca9 2023-02-08 23:53:56,908 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/ffb9382e1ff74e4881e941dc21a18864 2023-02-08 23:53:56,915 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/ffb9382e1ff74e4881e941dc21a18864 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/ffb9382e1ff74e4881e941dc21a18864 2023-02-08 23:53:56,921 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/ffb9382e1ff74e4881e941dc21a18864, entries=1, sequenceid=13, filesize=4.9 K 2023-02-08 23:53:56,922 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 843ms, sequenceid=13, compaction requested=true 2023-02-08 23:53:56,922 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:53:56,922 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:56,922 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:56,922 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:56,922 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,922 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:53:56,922 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:53:56,924 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,924 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:56,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:56,924 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,924 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:56,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:56,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:56,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:53:56,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69219869[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:56,926 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:53:56,926 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,927 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,927 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:53:56,927 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:53:56,927 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,927 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:56,927 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:56,929 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:53:56,929 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:56,929 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,929 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:53:56,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:56,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:56,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:56,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:56,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:56,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:53:56,933 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:53:56,933 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:56,933 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:56,933 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,933 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:56,936 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,936 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:56,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:56,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,936 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:56,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:56,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:56,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:56,937 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:56,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,938 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:56,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,941 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,941 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,941 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,941 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:56,941 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:56,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:56,941 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,941 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:56,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:56,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:53:56,941 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:56,941 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:56,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:56,942 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:56,941 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:56,943 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:56,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:56,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:57,043 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:57,043 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:57,043 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:57,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:57,048 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:57,049 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:57,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:57,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:57,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,051 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:57,051 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:57,052 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:53:57,052 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:53:57,052 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,052 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,053 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:57,053 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:57,054 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,054 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:57,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,055 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,056 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:53:57,056 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,056 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:53:57,056 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:57,057 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:53:57,057 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:57,057 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:57,057 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:57,057 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:53:57,057 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:57,473 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205948dd3f287e44e5c89e3349e0ea484f1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205948dd3f287e44e5c89e3349e0ea484f1 2023-02-08 23:53:57,879 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/7d377b2203b0474bbe9c6bde95fa71f7 2023-02-08 23:53:57,887 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/7d377b2203b0474bbe9c6bde95fa71f7 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/7d377b2203b0474bbe9c6bde95fa71f7 2023-02-08 23:53:57,893 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/7d377b2203b0474bbe9c6bde95fa71f7, entries=1, sequenceid=17, filesize=4.9 K 2023-02-08 23:53:57,894 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 837ms, sequenceid=17, compaction requested=true 2023-02-08 23:53:57,894 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:53:57,894 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:57,894 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:57,894 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:57,894 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,894 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:53:57,894 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:53:57,896 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,896 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:57,897 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,897 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:57,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:53:57,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d539cd6[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:57,898 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:53:57,899 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,900 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,900 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:53:57,900 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:53:57,900 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:57,900 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,900 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:57,902 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:53:57,902 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:57,902 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,902 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:53:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:57,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:57,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:53:57,906 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:53:57,906 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:57,906 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:57,906 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,906 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:57,907 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,907 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:57,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,907 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:57,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:57,908 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:57,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,909 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:57,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,912 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,912 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:57,912 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,912 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:57,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:53:57,912 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:57,912 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:57,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:57,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:57,912 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:57,913 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:57,913 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,913 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:57,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:57,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:57,913 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:57,913 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:58,013 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:58,014 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:58,014 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:58,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:58,020 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:58,020 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:58,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:58,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:58,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,023 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:58,023 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:58,024 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:53:58,024 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:53:58,024 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,024 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,026 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:58,026 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:58,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:58,026 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,026 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:58,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:58,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,027 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,027 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:53:58,028 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,028 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:53:58,028 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:58,028 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:53:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:58,029 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:58,029 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:58,029 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:53:58,029 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:58,049 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122853750c71574c42cdb6800d36596f149a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122853750c71574c42cdb6800d36596f149a 2023-02-08 23:53:58,061 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/bca1050c306e459482ffaf0607e20556 2023-02-08 23:53:58,068 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/bca1050c306e459482ffaf0607e20556 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/bca1050c306e459482ffaf0607e20556 2023-02-08 23:53:58,074 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/bca1050c306e459482ffaf0607e20556, entries=1, sequenceid=21, filesize=4.9 K 2023-02-08 23:53:58,075 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 46ms, sequenceid=21, compaction requested=true 2023-02-08 23:53:58,075 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:53:58,075 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:58,075 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:58,075 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:58,075 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,075 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:53:58,075 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:53:58,079 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:58,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:58,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:58,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:58,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:53:58,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74accfd2[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:58,081 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:53:58,081 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,082 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,082 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:53:58,083 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:53:58,083 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:58,083 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,083 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:58,084 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:53:58,084 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,084 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:58,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:58,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:58,084 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:53:58,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:58,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:58,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:53:58,087 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:53:58,087 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:58,087 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:58,087 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,088 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:58,092 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,092 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,092 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:58,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:58,094 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,095 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:58,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,099 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,099 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:58,099 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:58,099 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,099 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:58,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:58,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:58,099 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:58,099 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,100 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,100 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,100 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:58,100 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:58,100 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:58,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:58,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:58,200 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:58,201 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:58,201 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:58,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:58,206 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:58,207 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:58,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:58,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:58,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,224 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:58,224 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:58,225 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:53:58,225 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:53:58,226 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,226 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,228 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:58,228 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:58,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:58,228 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,229 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:58,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:58,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,234 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:53:58,234 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,234 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:53:58,234 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:58,235 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:53:58,240 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:58,240 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:58,240 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:58,241 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:53:58,241 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:58,276 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123150436467b0c04e27af642bbd5446af0d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123150436467b0c04e27af642bbd5446af0d 2023-02-08 23:53:58,283 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/e666cd4e229d464b9de624b7242d34cc 2023-02-08 23:53:58,290 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/e666cd4e229d464b9de624b7242d34cc as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/e666cd4e229d464b9de624b7242d34cc 2023-02-08 23:53:58,296 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/e666cd4e229d464b9de624b7242d34cc, entries=1, sequenceid=25, filesize=4.9 K 2023-02-08 23:53:58,297 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 56ms, sequenceid=25, compaction requested=true 2023-02-08 23:53:58,297 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:53:58,297 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:58,297 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:58,297 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:58,297 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,297 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:53:58,297 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:53:58,301 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,301 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:58,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:58,301 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:58,301 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:58,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:58,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:58,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:53:58,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e860608[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:58,303 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:53:58,303 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,304 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:53:58,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:53:58,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:58,305 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,305 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:58,306 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,306 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:53:58,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:58,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:58,306 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:58,306 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:53:58,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:58,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:58,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:58,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:58,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:53:58,316 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:53:58,316 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:58,316 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:58,316 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,316 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:58,318 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,318 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,318 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:58,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:58,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:58,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:58,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:58,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:58,319 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:58,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,321 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:58,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,325 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,325 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:58,325 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,325 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:58,325 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,325 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:58,325 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:58,325 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:58,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:58,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:58,325 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,326 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,326 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:58,326 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:58,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:58,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:58,426 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:58,427 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:58,427 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:58,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:58,434 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:58,434 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:58,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:58,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:58,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,437 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:58,437 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:58,438 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:53:58,438 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:53:58,439 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,439 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,440 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:58,440 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:58,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:58,440 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:58,440 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:58,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:58,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,441 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,444 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:53:58,444 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:58,444 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:53:58,444 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:58,444 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:58,448 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:58,448 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:58,448 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:53:58,448 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:58,870 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601012b23d95cd4dc444f8f9f952569e16821 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601012b23d95cd4dc444f8f9f952569e16821 2023-02-08 23:53:59,276 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/3bc7db3e2d474345be9edf4b7ca66ee4 2023-02-08 23:53:59,283 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/3bc7db3e2d474345be9edf4b7ca66ee4 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/3bc7db3e2d474345be9edf4b7ca66ee4 2023-02-08 23:53:59,288 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/3bc7db3e2d474345be9edf4b7ca66ee4, entries=1, sequenceid=29, filesize=4.9 K 2023-02-08 23:53:59,289 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 841ms, sequenceid=29, compaction requested=true 2023-02-08 23:53:59,289 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:53:59,289 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:59,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:53:59,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:53:59,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:53:59,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:53:59,292 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,292 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:59,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:59,292 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,292 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:53:59,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:59,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:59,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:59,293 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:53:59,293 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b012361[Count = 0] remaining members to acquire global barrier 2023-02-08 23:53:59,293 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:53:59,294 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,294 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,295 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,295 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:53:59,295 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:53:59,295 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:53:59,295 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:53:59,296 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:53:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:59,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:59,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:53:59,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:59,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:53:59,299 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:53:59,299 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:53:59,299 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:53:59,299 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:53:59,299 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,299 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:53:59,301 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,301 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,301 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:59,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:59,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:59,302 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:53:59,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:53:59,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:53:59,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,305 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:53:59,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:53:59,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,308 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,308 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:59,308 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:59,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:59,308 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:53:59,308 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,308 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:53:59,308 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:53:59,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:59,309 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:53:59,309 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,309 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,309 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:53:59,309 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:53:59,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:53:59,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:53:59,410 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:53:59,410 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:59,410 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:59,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:53:59,415 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:53:59,415 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:53:59,415 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:53:59,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:53:59,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,417 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:53:59,417 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:59,418 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:53:59,418 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:53:59,418 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,418 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,419 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:53:59,419 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:53:59,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:53:59,420 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:53:59,420 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:53:59,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,420 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:53:59,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,420 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,421 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:53:59,421 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:53:59,421 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:53:59,421 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:53:59,421 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:53:59,421 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:53:59,421 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:53:59,421 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:53:59,422 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:53:59,422 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:53:59,838 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f3120fab193a4cc0b02657c984fec21c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103f3120fab193a4cc0b02657c984fec21c 2023-02-08 23:54:00,244 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/c951164ec4d64e19ae468fc5ca696a1a 2023-02-08 23:54:00,251 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/c951164ec4d64e19ae468fc5ca696a1a as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/c951164ec4d64e19ae468fc5ca696a1a 2023-02-08 23:54:00,256 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/c951164ec4d64e19ae468fc5ca696a1a, entries=1, sequenceid=33, filesize=4.9 K 2023-02-08 23:54:00,257 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 835ms, sequenceid=33, compaction requested=true 2023-02-08 23:54:00,257 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:54:00,257 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:54:00,257 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:00,257 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:00,257 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,257 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:54:00,257 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:54:00,259 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,259 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:00,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:00,259 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,259 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:00,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:00,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:00,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:00,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:54:00,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a75a1fb[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:00,260 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:00,261 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,262 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,262 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:00,262 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:54:00,262 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:00,262 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,262 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:00,270 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:54:00,270 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,270 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:00,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:00,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:00,270 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:54:00,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:00,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:00,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:00,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:00,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:00,273 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:54:00,273 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:00,273 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:00,273 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,273 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:00,275 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,275 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:00,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:00,275 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:00,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,276 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:00,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:00,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:00,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:00,277 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:00,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,279 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,279 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:00,279 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:00,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:00,280 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:00,280 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:00,279 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:54:00,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:54:00,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:00,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:00,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:00,280 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,281 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,281 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,281 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,281 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:00,281 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:00,381 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:00,381 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:00,381 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:00,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:00,387 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:00,387 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:00,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:00,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:00,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,397 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:54:00,397 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:00,397 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:54:00,397 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:00,398 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,398 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,399 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,399 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:00,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:00,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:00,400 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:00,400 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:00,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:00,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:54:00,401 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:00,401 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:54:00,401 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:00,401 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:54:00,401 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:00,401 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:00,401 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:54:00,402 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:54:00,402 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:00,825 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079e282527540a4b4c9ad9f566bbe46bca to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509079e282527540a4b4c9ad9f566bbe46bca 2023-02-08 23:54:01,231 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/5b40416645eb4ea0ab23b7d65854a8b9 2023-02-08 23:54:01,238 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/5b40416645eb4ea0ab23b7d65854a8b9 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/5b40416645eb4ea0ab23b7d65854a8b9 2023-02-08 23:54:01,243 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/5b40416645eb4ea0ab23b7d65854a8b9, entries=1, sequenceid=37, filesize=4.9 K 2023-02-08 23:54:01,244 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 842ms, sequenceid=37, compaction requested=true 2023-02-08 23:54:01,244 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:54:01,244 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:54:01,244 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:01,244 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:01,244 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,244 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:54:01,244 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:54:01,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,246 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:01,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:01,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:01,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:01,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:01,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:01,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:54:01,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8352f38[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:01,248 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:01,248 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,250 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,250 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,250 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:01,250 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:01,250 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:54:01,250 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:01,252 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,252 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:01,252 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:01,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:01,252 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:54:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:01,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:01,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:01,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:01,255 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:54:01,255 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:01,255 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:01,255 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,255 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:01,257 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,257 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:01,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,257 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:01,257 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:01,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:01,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,259 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:01,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,261 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,262 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:01,262 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:01,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:54:01,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:01,262 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:01,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:01,262 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:01,262 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,262 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,263 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:01,263 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,263 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:01,263 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:01,263 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:01,363 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:01,363 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:01,363 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:01,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:01,369 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:01,369 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:01,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:01,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:01,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,371 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:54:01,371 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:01,372 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:54:01,372 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:01,373 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,373 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,374 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,374 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:01,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:01,374 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:01,374 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:01,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,376 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:54:01,376 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:01,376 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:54:01,376 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:01,377 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:54:01,377 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:01,377 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:01,377 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:54:01,377 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. started... 2023-02-08 23:54:01,378 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b83370f2c21546d39bae07b7ba35fa0 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:01,795 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112098c9df6ccb2441f8bd7a68f306edcea4 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112098c9df6ccb2441f8bd7a68f306edcea4 2023-02-08 23:54:02,202 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/4cae7fdeef9c478aab0baeac0e7b8360 2023-02-08 23:54:02,211 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/4cae7fdeef9c478aab0baeac0e7b8360 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/4cae7fdeef9c478aab0baeac0e7b8360 2023-02-08 23:54:02,222 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/4cae7fdeef9c478aab0baeac0e7b8360, entries=1, sequenceid=41, filesize=4.9 K 2023-02-08 23:54:02,223 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9b83370f2c21546d39bae07b7ba35fa0 in 845ms, sequenceid=41, compaction requested=true 2023-02-08 23:54:02,223 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:54:02,223 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:54:02,224 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:02,224 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:02,224 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,224 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:54:02,224 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:54:02,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,226 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:02,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:02,227 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:02,227 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:02,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:02,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:02,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:54:02,229 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60dce70[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:02,229 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:02,229 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,230 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,230 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:02,230 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:54:02,230 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:02,232 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,232 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:02,235 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,235 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:54:02,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:02,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:02,235 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:02,235 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:54:02,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:02,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:02,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:02,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:02,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:02,238 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:54:02,238 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:02,238 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:02,238 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,238 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:02,239 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,239 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,239 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:02,240 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:02,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:02,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:02,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,242 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:02,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,245 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:02,245 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:02,245 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:02,245 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:02,245 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 23:54:02,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:02,247 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,247 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:02,247 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,247 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,247 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,247 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:54:02,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:02,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:02,248 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:02,248 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:02,348 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:02,348 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:02,348 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:02,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:02,351 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-08 23:54:02,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:54:02,354 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:54:02,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:02,355 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901042355 2023-02-08 23:54:02,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:54:02,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-08 23:54:02,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:54:02,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x0bed65dc to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:02,375 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c0ddf86, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:02,376 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:54:02,837 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010366561bea77044f97bf84b2a1ce9d5576 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010366561bea77044f97bf84b2a1ce9d5576 2023-02-08 23:54:02,838 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:54:02,853 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:02,855 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46286, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:02,863 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:02,863 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:02,863 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:54:02,865 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43564, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:54:02,877 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/127b00a13e3e49ee864f2d37d229573c first=Optional[r5] last=Optional[r6] 2023-02-08 23:54:02,877 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/127b00a13e3e49ee864f2d37d229573c}] 2023-02-08 23:54:02,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/127b00a13e3e49ee864f2d37d229573c for inclusion in 9b83370f2c21546d39bae07b7ba35fa0/family1 2023-02-08 23:54:02,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-08 23:54:02,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:02,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:54:02,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/127b00a13e3e49ee864f2d37d229573c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4nkq5kl3p9kglg4k4h0bgq970jdj7ob6qj3238dssfvnge9no1c2br5j1018nqrh/family1/127b00a13e3e49ee864f2d37d229573c 2023-02-08 23:54:02,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4nkq5kl3p9kglg4k4h0bgq970jdj7ob6qj3238dssfvnge9no1c2br5j1018nqrh/family1/127b00a13e3e49ee864f2d37d229573c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/de5e89012f1b48cfb6e752f7c1b463c9_SeqId_44_ 2023-02-08 23:54:02,893 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4nkq5kl3p9kglg4k4h0bgq970jdj7ob6qj3238dssfvnge9no1c2br5j1018nqrh/family1/127b00a13e3e49ee864f2d37d229573c into 9b83370f2c21546d39bae07b7ba35fa0/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/de5e89012f1b48cfb6e752f7c1b463c9_SeqId_44_ - updating store file list. 2023-02-08 23:54:02,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/de5e89012f1b48cfb6e752f7c1b463c9_SeqId_44_ into 9b83370f2c21546d39bae07b7ba35fa0/family1 2023-02-08 23:54:02,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4nkq5kl3p9kglg4k4h0bgq970jdj7ob6qj3238dssfvnge9no1c2br5j1018nqrh/family1/127b00a13e3e49ee864f2d37d229573c into 9b83370f2c21546d39bae07b7ba35fa0/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/de5e89012f1b48cfb6e752f7c1b463c9_SeqId_44_) 2023-02-08 23:54:02,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4nkq5kl3p9kglg4k4h0bgq970jdj7ob6qj3238dssfvnge9no1c2br5j1018nqrh/family1/127b00a13e3e49ee864f2d37d229573c 2023-02-08 23:54:02,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 23:54:02,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 23:54:02,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 9b83370f2c21546d39bae07b7ba35fa0/family1 is initiating minor compaction 2023-02-08 23:54:02,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1c94503b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:02,912 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9b83370f2c21546d39bae07b7ba35fa0/family1 in testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:54:02,913 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/d98e59e0efc749d08cea4cf5ba455e80, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/ffb9382e1ff74e4881e941dc21a18864, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/7d377b2203b0474bbe9c6bde95fa71f7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/bca1050c306e459482ffaf0607e20556, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/e666cd4e229d464b9de624b7242d34cc, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/3bc7db3e2d474345be9edf4b7ca66ee4, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/c951164ec4d64e19ae468fc5ca696a1a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/5b40416645eb4ea0ab23b7d65854a8b9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/4cae7fdeef9c478aab0baeac0e7b8360, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/de5e89012f1b48cfb6e752f7c1b463c9_SeqId_44_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp, totalSize=49.7 K 2023-02-08 23:54:02,913 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting d98e59e0efc749d08cea4cf5ba455e80, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 23:54:02,913 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting ffb9382e1ff74e4881e941dc21a18864, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 23:54:02,913 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 7d377b2203b0474bbe9c6bde95fa71f7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 23:54:02,913 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting bca1050c306e459482ffaf0607e20556, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 23:54:02,913 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting e666cd4e229d464b9de624b7242d34cc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 23:54:02,913 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 3bc7db3e2d474345be9edf4b7ca66ee4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 23:54:02,913 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting c951164ec4d64e19ae468fc5ca696a1a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 23:54:02,914 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 5b40416645eb4ea0ab23b7d65854a8b9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 23:54:02,914 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 4cae7fdeef9c478aab0baeac0e7b8360, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 23:54:02,914 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting de5e89012f1b48cfb6e752f7c1b463c9_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 23:54:02,919 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:54:02,920 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:02,922 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601012b23d95cd4dc444f8f9f952569e16821 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601012b23d95cd4dc444f8f9f952569e16821 2023-02-08 23:54:02,924 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103f3120fab193a4cc0b02657c984fec21c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103f3120fab193a4cc0b02657c984fec21c 2023-02-08 23:54:02,924 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-08 23:54:02,924 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-08 23:54:02,925 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:54:02,925 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:54:02,977 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9b83370f2c21546d39bae07b7ba35fa0#family1#compaction#22 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:03,790 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231e1df54343f7640378b281dc34747f92a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231e1df54343f7640378b281dc34747f92a 2023-02-08 23:54:03,791 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:54:03,807 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/589bfce41b61415e8e04ca864d0a3afb as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/589bfce41b61415e8e04ca864d0a3afb 2023-02-08 23:54:03,814 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:03,815 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:03,815 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 9b83370f2c21546d39bae07b7ba35fa0/family1 of 9b83370f2c21546d39bae07b7ba35fa0 into 589bfce41b61415e8e04ca864d0a3afb(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:03,816 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:54:03,816 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675900442912; duration=0sec 2023-02-08 23:54:03,816 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:03,838 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/f365e817c32b438186402c303e80b786 first=Optional[r1] last=Optional[r4] 2023-02-08 23:54:03,839 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/f365e817c32b438186402c303e80b786}] 2023-02-08 23:54:03,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/f365e817c32b438186402c303e80b786 for inclusion in 9b83370f2c21546d39bae07b7ba35fa0/family1 2023-02-08 23:54:03,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-08 23:54:03,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:03,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:54:03,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/f365e817c32b438186402c303e80b786 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ii96t5b199ubom4d6ck6dj08rpp9r9jn1s4k1sepn6k6d68l40126oq0ff991keh/family1/f365e817c32b438186402c303e80b786 2023-02-08 23:54:03,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ii96t5b199ubom4d6ck6dj08rpp9r9jn1s4k1sepn6k6d68l40126oq0ff991keh/family1/f365e817c32b438186402c303e80b786 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/8439f6af0b1d412baf251a21a524ed39_SeqId_47_ 2023-02-08 23:54:03,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ii96t5b199ubom4d6ck6dj08rpp9r9jn1s4k1sepn6k6d68l40126oq0ff991keh/family1/f365e817c32b438186402c303e80b786 into 9b83370f2c21546d39bae07b7ba35fa0/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/8439f6af0b1d412baf251a21a524ed39_SeqId_47_ - updating store file list. 2023-02-08 23:54:03,862 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/d98e59e0efc749d08cea4cf5ba455e80, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/ffb9382e1ff74e4881e941dc21a18864, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/7d377b2203b0474bbe9c6bde95fa71f7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/bca1050c306e459482ffaf0607e20556, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/e666cd4e229d464b9de624b7242d34cc, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/3bc7db3e2d474345be9edf4b7ca66ee4, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/c951164ec4d64e19ae468fc5ca696a1a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/5b40416645eb4ea0ab23b7d65854a8b9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/4cae7fdeef9c478aab0baeac0e7b8360, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/de5e89012f1b48cfb6e752f7c1b463c9_SeqId_44_] to archive 2023-02-08 23:54:03,864 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:03,866 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/d98e59e0efc749d08cea4cf5ba455e80 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/d98e59e0efc749d08cea4cf5ba455e80 2023-02-08 23:54:03,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/8439f6af0b1d412baf251a21a524ed39_SeqId_47_ into 9b83370f2c21546d39bae07b7ba35fa0/family1 2023-02-08 23:54:03,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ii96t5b199ubom4d6ck6dj08rpp9r9jn1s4k1sepn6k6d68l40126oq0ff991keh/family1/f365e817c32b438186402c303e80b786 into 9b83370f2c21546d39bae07b7ba35fa0/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/8439f6af0b1d412baf251a21a524ed39_SeqId_47_) 2023-02-08 23:54:03,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ii96t5b199ubom4d6ck6dj08rpp9r9jn1s4k1sepn6k6d68l40126oq0ff991keh/family1/f365e817c32b438186402c303e80b786 2023-02-08 23:54:03,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 23:54:03,867 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/ffb9382e1ff74e4881e941dc21a18864 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/ffb9382e1ff74e4881e941dc21a18864 2023-02-08 23:54:03,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 23:54:03,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 9b83370f2c21546d39bae07b7ba35fa0/family1 is initiating minor compaction (all files) 2023-02-08 23:54:03,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@65177d05; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:03,868 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9b83370f2c21546d39bae07b7ba35fa0/family1 in testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:54:03,869 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/50706af476e54c85be9147f41bebb4ae, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/589bfce41b61415e8e04ca864d0a3afb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/8439f6af0b1d412baf251a21a524ed39_SeqId_47_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp, totalSize=178.1 K 2023-02-08 23:54:03,869 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/7d377b2203b0474bbe9c6bde95fa71f7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/7d377b2203b0474bbe9c6bde95fa71f7 2023-02-08 23:54:03,869 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 50706af476e54c85be9147f41bebb4ae, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838754 2023-02-08 23:54:03,870 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 589bfce41b61415e8e04ca864d0a3afb, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238754 2023-02-08 23:54:03,870 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 8439f6af0b1d412baf251a21a524ed39_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965238754 2023-02-08 23:54:03,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/bca1050c306e459482ffaf0607e20556 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/bca1050c306e459482ffaf0607e20556 2023-02-08 23:54:03,871 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:54:03,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/e666cd4e229d464b9de624b7242d34cc to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/e666cd4e229d464b9de624b7242d34cc 2023-02-08 23:54:03,872 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:03,875 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201dc3fc3be5ee64ed5b9646f416ea18ca9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201dc3fc3be5ee64ed5b9646f416ea18ca9 2023-02-08 23:54:03,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/3bc7db3e2d474345be9edf4b7ca66ee4 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/3bc7db3e2d474345be9edf4b7ca66ee4 2023-02-08 23:54:03,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/c951164ec4d64e19ae468fc5ca696a1a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/c951164ec4d64e19ae468fc5ca696a1a 2023-02-08 23:54:03,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/5b40416645eb4ea0ab23b7d65854a8b9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/5b40416645eb4ea0ab23b7d65854a8b9 2023-02-08 23:54:03,880 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205948dd3f287e44e5c89e3349e0ea484f1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205948dd3f287e44e5c89e3349e0ea484f1 2023-02-08 23:54:03,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/4cae7fdeef9c478aab0baeac0e7b8360 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/4cae7fdeef9c478aab0baeac0e7b8360 2023-02-08 23:54:03,883 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122853750c71574c42cdb6800d36596f149a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015122853750c71574c42cdb6800d36596f149a 2023-02-08 23:54:03,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/de5e89012f1b48cfb6e752f7c1b463c9_SeqId_44_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/de5e89012f1b48cfb6e752f7c1b463c9_SeqId_44_ 2023-02-08 23:54:03,884 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123150436467b0c04e27af642bbd5446af0d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123150436467b0c04e27af642bbd5446af0d 2023-02-08 23:54:03,884 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-08 23:54:03,885 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-08 23:54:03,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:54:03,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:54:03,912 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9b83370f2c21546d39bae07b7ba35fa0#family1#compaction#23 average throughput is 5.76 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:04,349 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307f39c0c2b8bd41e683ec2ac6fa5c4108 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511307f39c0c2b8bd41e683ec2ac6fa5c4108 2023-02-08 23:54:04,350 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:54:04,363 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/.tmp/family1/898b5fc1ccf74034a238ecbf19b0c271 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/898b5fc1ccf74034a238ecbf19b0c271 2023-02-08 23:54:04,367 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:04,367 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:04,369 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 9b83370f2c21546d39bae07b7ba35fa0/family1 of 9b83370f2c21546d39bae07b7ba35fa0 into 898b5fc1ccf74034a238ecbf19b0c271(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:04,369 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:54:04,369 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1675900443868; duration=0sec 2023-02-08 23:54:04,369 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:04,375 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6a9a9d660917435f9bfdb6abfaa826a9 first=Optional[r00] last=Optional[r8] 2023-02-08 23:54:04,376 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6a9a9d660917435f9bfdb6abfaa826a9}] 2023-02-08 23:54:04,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6a9a9d660917435f9bfdb6abfaa826a9 for inclusion in 9b83370f2c21546d39bae07b7ba35fa0/family1 2023-02-08 23:54:04,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-08 23:54:04,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:04,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 9b83370f2c21546d39bae07b7ba35fa0: 2023-02-08 23:54:04,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/6a9a9d660917435f9bfdb6abfaa826a9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__a1u1ee6pce2cled3r061jhl4nj7euisnm1lt4qmheb2pqndbtb2008325g9msqb7/family1/6a9a9d660917435f9bfdb6abfaa826a9 2023-02-08 23:54:04,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__a1u1ee6pce2cled3r061jhl4nj7euisnm1lt4qmheb2pqndbtb2008325g9msqb7/family1/6a9a9d660917435f9bfdb6abfaa826a9 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/f8c9b526977d47ecada5bced6c4ca804_SeqId_50_ 2023-02-08 23:54:04,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__a1u1ee6pce2cled3r061jhl4nj7euisnm1lt4qmheb2pqndbtb2008325g9msqb7/family1/6a9a9d660917435f9bfdb6abfaa826a9 into 9b83370f2c21546d39bae07b7ba35fa0/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/f8c9b526977d47ecada5bced6c4ca804_SeqId_50_ - updating store file list. 2023-02-08 23:54:04,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/f8c9b526977d47ecada5bced6c4ca804_SeqId_50_ into 9b83370f2c21546d39bae07b7ba35fa0/family1 2023-02-08 23:54:04,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__a1u1ee6pce2cled3r061jhl4nj7euisnm1lt4qmheb2pqndbtb2008325g9msqb7/family1/6a9a9d660917435f9bfdb6abfaa826a9 into 9b83370f2c21546d39bae07b7ba35fa0/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/f8c9b526977d47ecada5bced6c4ca804_SeqId_50_) 2023-02-08 23:54:04,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithMonthlyPolicy__a1u1ee6pce2cled3r061jhl4nj7euisnm1lt4qmheb2pqndbtb2008325g9msqb7/family1/6a9a9d660917435f9bfdb6abfaa826a9 2023-02-08 23:54:04,391 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:54:04,392 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:04,394 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112098c9df6ccb2441f8bd7a68f306edcea4 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112098c9df6ccb2441f8bd7a68f306edcea4 2023-02-08 23:54:04,395 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511287b297ff218ff4eed8e7d9afb8a91872c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511287b297ff218ff4eed8e7d9afb8a91872c 2023-02-08 23:54:04,396 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130372443120f1146aa8a603c1c81e89ded to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130372443120f1146aa8a603c1c81e89ded 2023-02-08 23:54:04,396 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-08 23:54:04,396 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 23:54:04,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:54:04,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:54:04,397 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:54:04,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:54:04,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:04,398 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:54:04,400 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.0460 sec 2023-02-08 23:54:04,461 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/50706af476e54c85be9147f41bebb4ae, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/589bfce41b61415e8e04ca864d0a3afb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/8439f6af0b1d412baf251a21a524ed39_SeqId_47_] to archive 2023-02-08 23:54:04,462 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:04,466 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/50706af476e54c85be9147f41bebb4ae to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/50706af476e54c85be9147f41bebb4ae 2023-02-08 23:54:04,468 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/589bfce41b61415e8e04ca864d0a3afb to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/589bfce41b61415e8e04ca864d0a3afb 2023-02-08 23:54:04,471 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/8439f6af0b1d412baf251a21a524ed39_SeqId_47_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/family1/8439f6af0b1d412baf251a21a524ed39_SeqId_47_ 2023-02-08 23:54:04,520 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=253 (was 249) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54774@0x0bed65dc sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1999313532.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:55674 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-494294014_15 at /127.0.0.1:55716 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d5cbb49-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:55852 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:55768 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54774@0x0bed65dc-SendThread(127.0.0.1:54774) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:55788 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:55802 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:33840 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:54774@0x0bed65dc-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:33834 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d5cbb49-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:55826 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d5cbb49-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:55690 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44390 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/cluster_0361639e-2335-beea-2849-07295ee226f0/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:44612 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d5cbb49-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d5cbb49-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:55818 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d5cbb49-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d5cbb49-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_382700349_15 at /127.0.0.1:55704 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=600 (was 593) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=286 (was 284) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3667 (was 4080) 2023-02-08 23:54:04,535 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=253, OpenFileDescriptor=600, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=168, AvailableMemoryMB=3666 2023-02-08 23:54:04,536 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:04,536 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:04,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 23:54:04,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:04,544 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:04,545 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:04,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-08 23:54:04,546 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:04,546 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:04,547 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:54:04,548 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a empty. 2023-02-08 23:54:04,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 23:54:04,549 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:54:04,549 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-08 23:54:04,649 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:04,649 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:04,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 23:54:04,851 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:04,851 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:04,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 23:54:04,966 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:04,968 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => a9e06470572e5280a9822eb7682ca46a, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:54:05,152 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,152 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 23:54:05,376 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:05,376 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing a9e06470572e5280a9822eb7682ca46a, disabling compactions & flushes 2023-02-08 23:54:05,376 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:05,376 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:05,376 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. after waiting 0 ms 2023-02-08 23:54:05,376 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:05,376 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:05,376 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for a9e06470572e5280a9822eb7682ca46a: Waiting for close lock at 1675900445376Disabling compacts and flushes for region at 1675900445376Disabling writes for close at 1675900445376Writing region close event to WAL at 1675900445376Closed at 1675900445376 2023-02-08 23:54:05,379 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:05,380 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900445379"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900445379"}]},"ts":"1675900445379"} 2023-02-08 23:54:05,381 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:05,383 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:05,383 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900445383"}]},"ts":"1675900445383"} 2023-02-08 23:54:05,384 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-08 23:54:05,390 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=a9e06470572e5280a9822eb7682ca46a, ASSIGN}] 2023-02-08 23:54:05,392 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=a9e06470572e5280a9822eb7682ca46a, ASSIGN 2023-02-08 23:54:05,393 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=a9e06470572e5280a9822eb7682ca46a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:54:05,544 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=a9e06470572e5280a9822eb7682ca46a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:05,544 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900445544"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900445544"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900445544"}]},"ts":"1675900445544"} 2023-02-08 23:54:05,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure a9e06470572e5280a9822eb7682ca46a, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:05,653 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:05,653 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:05,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 23:54:05,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:05,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a9e06470572e5280a9822eb7682ca46a, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:05,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:54:05,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:05,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:54:05,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:54:05,704 INFO [StoreOpener-a9e06470572e5280a9822eb7682ca46a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:54:05,706 DEBUG [StoreOpener-a9e06470572e5280a9822eb7682ca46a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1 2023-02-08 23:54:05,706 DEBUG [StoreOpener-a9e06470572e5280a9822eb7682ca46a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1 2023-02-08 23:54:05,706 INFO [StoreOpener-a9e06470572e5280a9822eb7682ca46a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9e06470572e5280a9822eb7682ca46a columnFamilyName family1 2023-02-08 23:54:05,708 INFO [StoreOpener-a9e06470572e5280a9822eb7682ca46a-1] regionserver.HStore(310): Store=a9e06470572e5280a9822eb7682ca46a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:05,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:54:05,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:54:05,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:54:05,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:05,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a9e06470572e5280a9822eb7682ca46a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61469450, jitterRate=-0.08403381705284119}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:05,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a9e06470572e5280a9822eb7682ca46a: Running coprocessor pre-open hook at 1675900445703Writing region info on filesystem at 1675900445703Initializing all the Stores at 1675900445704 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675900445704Cleaning up temporary data from old regions at 1675900445710 (+6 ms)Cleaning up detritus from prior splits at 1675900445710Running coprocessor post-open hooks at 1675900445717 (+7 ms)Region opened successfully at 1675900445717 2023-02-08 23:54:05,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a., pid=25, masterSystemTime=1675900445698 2023-02-08 23:54:05,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:05,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:05,720 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=a9e06470572e5280a9822eb7682ca46a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:05,720 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900445720"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900445720"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900445720"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900445720"}]},"ts":"1675900445720"} 2023-02-08 23:54:05,724 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-08 23:54:05,724 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure a9e06470572e5280a9822eb7682ca46a, server=jenkins-hbase4.apache.org,42703,1675900414768 in 176 msec 2023-02-08 23:54:05,726 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-08 23:54:05,726 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=a9e06470572e5280a9822eb7682ca46a, ASSIGN in 334 msec 2023-02-08 23:54:05,727 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:05,727 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900445727"}]},"ts":"1675900445727"} 2023-02-08 23:54:05,728 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-08 23:54:05,731 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:05,732 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1920 sec 2023-02-08 23:54:05,924 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:06,655 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:06,655 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:06,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 23:54:06,656 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-08 23:54:06,682 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:06,682 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:06,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:06,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:06,688 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:06,688 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:06,688 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:06,688 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:06,688 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:06,688 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:06,690 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:06,690 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:06,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:06,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:06,690 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:06,690 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:06,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:06,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:06,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:06,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:06,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:06,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:06,691 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:06,691 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:06,691 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:06,692 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:06,692 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:06,692 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:06,692 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:06,693 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 23:54:06,716 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a94011e6d97349e6bee94b04a7f31fb0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130a94011e6d97349e6bee94b04a7f31fb0 2023-02-08 23:54:07,120 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/d399625b70ff442f85060ec0ad29c653 2023-02-08 23:54:07,127 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/d399625b70ff442f85060ec0ad29c653 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/d399625b70ff442f85060ec0ad29c653 2023-02-08 23:54:07,131 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/d399625b70ff442f85060ec0ad29c653, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-08 23:54:07,132 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 439ms, sequenceid=5, compaction requested=false 2023-02-08 23:54:07,133 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:07,133 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:07,133 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:07,133 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:07,133 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,133 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:07,133 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:07,135 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,135 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:07,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:07,135 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,135 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:07,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:07,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:07,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:07,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:07,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54c5fffb[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:07,136 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:07,136 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,138 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,138 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:07,138 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:07,138 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:07,139 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,139 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:07,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:07,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:07,140 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:07,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:07,142 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:07,142 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:07,142 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:07,142 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,142 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:07,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,144 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:07,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:07,144 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:07,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,145 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,147 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,147 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:07,147 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:07,147 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:07,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:07,147 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:07,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:07,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:07,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:07,147 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,148 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,148 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,148 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,148 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,148 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:07,148 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:07,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:07,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:07,248 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:07,248 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:07,248 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:07,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:07,253 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:07,253 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:07,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:07,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:07,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,256 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:07,256 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:07,256 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:07,256 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:07,256 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,256 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,262 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:07,262 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:07,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:07,263 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,263 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:07,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:07,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,264 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,264 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:07,264 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,264 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:07,264 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:07,264 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:07,264 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:07,264 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:07,264 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:07,264 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:07,264 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 23:54:07,679 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128bb3b7c238eaa41b1beadd22ceb973631 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128bb3b7c238eaa41b1beadd22ceb973631 2023-02-08 23:54:07,687 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/7034ffd7b6f340c6a87f9aacc9365262 2023-02-08 23:54:07,693 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/7034ffd7b6f340c6a87f9aacc9365262 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/7034ffd7b6f340c6a87f9aacc9365262 2023-02-08 23:54:07,698 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/7034ffd7b6f340c6a87f9aacc9365262, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 23:54:07,699 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 435ms, sequenceid=9, compaction requested=false 2023-02-08 23:54:07,699 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:07,699 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:07,699 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:07,700 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:07,700 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,700 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:07,700 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:07,701 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,701 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:07,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:07,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:07,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:07,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:07,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6308e6e1[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:07,703 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:07,703 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,704 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,705 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:07,705 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:07,705 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:07,705 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,705 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:07,707 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:07,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:07,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:07,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:07,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:07,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:07,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:07,710 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:07,710 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:07,710 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:07,710 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,710 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:07,713 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,713 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:07,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,713 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:07,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:07,713 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:07,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:07,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:07,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:07,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,714 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:07,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,717 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:07,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:07,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:07,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:07,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:07,717 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:07,718 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:07,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:07,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:07,818 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:07,818 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:07,818 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:07,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:07,823 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:07,823 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:07,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:07,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:07,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,826 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:07,826 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:07,826 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:07,826 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:07,826 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,826 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,828 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:07,828 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:07,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:07,828 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:07,828 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:07,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:07,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,829 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,829 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:07,829 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:07,829 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:07,829 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:07,829 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:07,830 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:07,830 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:07,830 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:07,830 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:07,830 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:07,850 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201bfc29d53eeed41cbb6c0794829bce8df to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201bfc29d53eeed41cbb6c0794829bce8df 2023-02-08 23:54:08,256 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/05684d359421467f9e9d55f2d8981724 2023-02-08 23:54:08,261 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/05684d359421467f9e9d55f2d8981724 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/05684d359421467f9e9d55f2d8981724 2023-02-08 23:54:08,267 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/05684d359421467f9e9d55f2d8981724, entries=1, sequenceid=13, filesize=4.9 K 2023-02-08 23:54:08,268 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 438ms, sequenceid=13, compaction requested=true 2023-02-08 23:54:08,268 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:08,268 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:08,268 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:08,268 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:08,268 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,268 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:08,268 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:08,270 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,270 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:08,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:08,270 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,270 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:08,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:08,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:08,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:08,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:08,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42654957[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:08,271 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:08,271 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,274 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,274 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:08,274 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:08,274 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,274 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:08,274 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:08,276 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,276 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:08,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:08,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:08,276 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:08,276 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:08,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:08,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:08,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:08,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:08,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:08,278 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:08,278 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:08,278 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:08,278 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,278 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:08,279 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,279 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:08,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:08,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,279 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:08,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:08,280 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:08,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:08,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:08,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,281 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:08,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:08,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:08,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:08,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:08,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:08,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,288 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:08,288 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,289 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:08,389 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:08,389 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,389 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:08,395 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,395 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:08,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:08,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,398 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:08,398 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:08,398 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:08,398 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:08,399 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,399 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,400 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,400 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:08,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:08,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:08,401 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,401 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:08,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,402 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:08,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:08,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:08,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:08,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:08,403 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:08,403 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:08,404 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:08,437 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054b05c1c0228b4ed99dbdd04e9673d845 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512054b05c1c0228b4ed99dbdd04e9673d845 2023-02-08 23:54:08,451 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/441c47d2b4a54e238ab08bb5c79ccddc 2023-02-08 23:54:08,460 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/441c47d2b4a54e238ab08bb5c79ccddc as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/441c47d2b4a54e238ab08bb5c79ccddc 2023-02-08 23:54:08,466 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/441c47d2b4a54e238ab08bb5c79ccddc, entries=1, sequenceid=17, filesize=4.9 K 2023-02-08 23:54:08,467 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 63ms, sequenceid=17, compaction requested=true 2023-02-08 23:54:08,467 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:08,467 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:08,467 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:08,467 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:08,467 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,467 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:08,467 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:08,469 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:08,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:08,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:08,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:08,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a737125[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:08,471 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:08,471 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,474 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,474 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:08,474 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:08,474 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:08,474 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,474 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:08,475 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:08,475 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,475 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:08,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,475 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:08,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:08,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:08,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:08,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:08,478 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:08,478 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:08,478 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:08,478 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,478 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:08,480 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:08,480 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,480 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:08,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:08,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:08,481 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,483 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,485 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,485 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:08,486 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:08,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:08,485 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:08,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:08,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:08,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:08,486 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,486 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:08,486 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:08,486 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:08,486 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,487 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,487 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,487 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:08,587 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:08,587 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,587 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:08,591 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,592 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:08,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:08,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,594 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:08,594 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:08,594 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:08,594 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:08,594 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,595 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,597 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:08,597 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:08,597 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,597 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:08,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:08,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:08,599 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:08,599 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:08,599 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:08,599 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:08,614 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228250fa08d806840a58687f4e44d9db493 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228250fa08d806840a58687f4e44d9db493 2023-02-08 23:54:08,624 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/c5588e2b916b4038a02370466149ff73 2023-02-08 23:54:08,633 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/c5588e2b916b4038a02370466149ff73 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c5588e2b916b4038a02370466149ff73 2023-02-08 23:54:08,640 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c5588e2b916b4038a02370466149ff73, entries=1, sequenceid=21, filesize=4.9 K 2023-02-08 23:54:08,641 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 42ms, sequenceid=21, compaction requested=true 2023-02-08 23:54:08,641 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:08,641 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:08,641 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:08,641 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:08,641 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,641 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:08,641 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:08,643 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,643 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:08,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:08,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:08,644 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,644 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:08,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:08,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:08,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:08,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51b269b7[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:08,646 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:08,646 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,647 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:08,647 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,648 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:08,649 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:08,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:08,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:08,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:08,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:08,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:08,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:08,653 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:08,653 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:08,653 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:08,653 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:08,653 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,653 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:08,656 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,656 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:08,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:08,656 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:08,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:08,657 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:08,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,659 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:08,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,662 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,662 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:08,662 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:08,662 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:08,662 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:08,662 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:08,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:08,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:08,662 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,662 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,663 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,663 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,663 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:08,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:08,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:08,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:08,663 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:08,763 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:08,763 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,764 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:08,768 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:08,769 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:08,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:08,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:08,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,771 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:08,771 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:08,771 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:08,771 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:08,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,773 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,773 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:08,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:08,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:08,774 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:08,774 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:08,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:08,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,775 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,779 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:08,779 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:08,779 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:08,779 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:08,779 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:08,779 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:08,779 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:08,779 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:08,779 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:08,779 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:08,799 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512310c0d6349b71f4a288369894f6d99d0f1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512310c0d6349b71f4a288369894f6d99d0f1 2023-02-08 23:54:09,205 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/4cc9a42b52854e8f894c6295408f1925 2023-02-08 23:54:09,210 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/4cc9a42b52854e8f894c6295408f1925 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/4cc9a42b52854e8f894c6295408f1925 2023-02-08 23:54:09,215 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/4cc9a42b52854e8f894c6295408f1925, entries=1, sequenceid=25, filesize=4.9 K 2023-02-08 23:54:09,216 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 437ms, sequenceid=25, compaction requested=true 2023-02-08 23:54:09,216 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:09,216 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:09,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:09,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:09,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:09,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:09,218 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,218 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,218 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,218 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:09,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:09,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75a9a28e[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:09,219 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:09,220 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,221 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,221 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:09,221 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:09,221 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:09,221 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,221 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:09,222 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,222 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:09,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,222 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,223 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:09,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:09,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:09,225 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:09,225 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:09,225 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:09,225 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,225 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:09,230 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,230 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,230 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,231 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,232 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,234 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,234 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,234 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,234 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,234 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,234 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,234 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,234 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,234 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:09,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:09,234 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:09,235 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:09,235 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:09,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,335 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:09,335 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,335 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:09,340 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,340 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:09,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:09,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,342 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:09,342 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,342 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:09,342 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:09,343 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,343 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,344 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,344 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,344 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,344 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:09,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:09,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,345 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,345 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:09,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,345 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:09,346 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,346 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:09,346 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:09,346 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:09,346 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:09,346 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:09,346 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:09,761 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101bdffea5679e74f1dbe5e15983e4aa92a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101bdffea5679e74f1dbe5e15983e4aa92a 2023-02-08 23:54:09,766 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/de6b8448ccca438a9582f105c68fcf73 2023-02-08 23:54:09,772 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/de6b8448ccca438a9582f105c68fcf73 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/de6b8448ccca438a9582f105c68fcf73 2023-02-08 23:54:09,777 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/de6b8448ccca438a9582f105c68fcf73, entries=1, sequenceid=29, filesize=4.9 K 2023-02-08 23:54:09,778 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 432ms, sequenceid=29, compaction requested=true 2023-02-08 23:54:09,778 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:09,778 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:09,778 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:09,778 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:09,778 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,778 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:09,778 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:09,780 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,780 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,781 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,781 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:09,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:09,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1dee629d[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:09,782 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:09,782 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,784 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,784 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:09,784 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:09,784 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:09,784 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,784 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:09,785 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,785 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:09,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,785 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,786 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:09,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:09,788 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:09,788 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:09,788 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:09,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:09,788 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,789 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:09,790 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,790 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:09,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:09,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,790 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:09,791 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,795 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:09,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:09,799 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:09,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:09,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:09,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:09,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:09,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:09,799 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,900 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:09,900 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,900 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:09,904 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:09,905 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:09,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:09,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:09,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,907 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:09,907 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,907 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:09,907 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:09,907 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,907 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,909 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:09,909 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:09,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:09,910 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:09,910 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:09,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:09,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:09,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:09,911 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:09,911 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:09,911 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:09,911 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:09,911 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:09,911 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:09,911 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:09,911 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:10,325 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103460e180ba74d4610a89582ab6bcc27a0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103460e180ba74d4610a89582ab6bcc27a0 2023-02-08 23:54:10,731 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/626bc9f38ec844e08413c373f0dd72ca 2023-02-08 23:54:10,737 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/626bc9f38ec844e08413c373f0dd72ca as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/626bc9f38ec844e08413c373f0dd72ca 2023-02-08 23:54:10,742 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/626bc9f38ec844e08413c373f0dd72ca, entries=1, sequenceid=33, filesize=4.9 K 2023-02-08 23:54:10,743 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 832ms, sequenceid=33, compaction requested=true 2023-02-08 23:54:10,743 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:10,743 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:10,743 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:10,743 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:10,743 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,743 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:10,743 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:10,745 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,745 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,745 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,745 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:10,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,747 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:10,747 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ca5d25c[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:10,747 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:10,747 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,748 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,748 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:10,748 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:10,748 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,748 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:10,748 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:10,751 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:10,751 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,751 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,751 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:10,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:10,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:10,753 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:10,753 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:10,753 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:10,753 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,753 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:10,755 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,755 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:10,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:10,755 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,755 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:10,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:10,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,756 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,758 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,758 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,758 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,758 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:10,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:10,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:10,758 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:10,758 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:10,758 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,759 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:10,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:10,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:10,759 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:10,759 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,759 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,759 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,859 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:10,859 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:10,859 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:10,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:10,864 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:10,864 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:10,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:10,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:10,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,867 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:10,867 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,867 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:10,867 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:10,867 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,867 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,868 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:10,868 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:10,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:10,869 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:10,869 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:10,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:10,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,870 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,870 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:10,870 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:10,870 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:10,870 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:10,870 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:10,870 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:10,870 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:10,870 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:10,870 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:10,870 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:11,237 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:54:11,284 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090789263b59dae44b6b8b1d7a74e6db9320 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015090789263b59dae44b6b8b1d7a74e6db9320 2023-02-08 23:54:11,690 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/c02605da9bab4a1fbc89078e4482b1db 2023-02-08 23:54:11,696 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/c02605da9bab4a1fbc89078e4482b1db as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c02605da9bab4a1fbc89078e4482b1db 2023-02-08 23:54:11,700 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c02605da9bab4a1fbc89078e4482b1db, entries=1, sequenceid=37, filesize=4.9 K 2023-02-08 23:54:11,701 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 831ms, sequenceid=37, compaction requested=true 2023-02-08 23:54:11,701 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:11,701 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:11,701 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:11,702 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:11,702 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,702 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:11,702 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:11,708 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,708 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:11,709 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,709 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e18f9ae[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:11,710 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:11,710 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,714 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,714 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:11,714 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,715 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:11,715 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:11,715 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:11,716 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,716 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:11,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:11,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:11,716 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:11,716 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:11,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:11,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:11,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:11,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:11,719 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:11,719 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:11,719 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:11,719 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:11,719 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,719 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:11,721 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,721 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:11,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:11,721 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:11,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:11,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:11,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:11,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,722 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:11,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,723 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:11,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,726 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,726 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:11,726 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:11,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:11,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:11,726 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,726 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:11,726 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:11,726 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:11,726 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,727 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:11,727 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:11,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:11,727 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:11,727 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,827 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:11,827 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:11,827 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:11,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:11,832 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:11,832 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:11,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:11,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:11,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,835 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:11,835 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:11,835 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:11,835 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:11,835 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,835 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,837 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:11,837 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:11,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:11,837 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:11,837 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:11,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,838 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:11,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,838 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,840 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:11,840 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:11,840 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:11,841 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. started... 2023-02-08 23:54:11,841 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e06470572e5280a9822eb7682ca46a 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:12,256 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e900f904495443c2ab0f2cd5b49ea74b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120e900f904495443c2ab0f2cd5b49ea74b 2023-02-08 23:54:12,662 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/79072536fc2d40dd803da57df73d4209 2023-02-08 23:54:12,667 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/79072536fc2d40dd803da57df73d4209 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/79072536fc2d40dd803da57df73d4209 2023-02-08 23:54:12,672 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/79072536fc2d40dd803da57df73d4209, entries=1, sequenceid=41, filesize=4.9 K 2023-02-08 23:54:12,673 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a9e06470572e5280a9822eb7682ca46a in 832ms, sequenceid=41, compaction requested=true 2023-02-08 23:54:12,673 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:12,673 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:12,673 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:12,673 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:12,673 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,673 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:12,673 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:12,674 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,674 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:12,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:12,675 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,675 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:12,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:12,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:12,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:12,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:12,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@561d9b31[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:12,676 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:12,676 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,678 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,678 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:12,678 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,678 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:12,678 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:12,678 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:12,680 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:12,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:12,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:12,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:12,682 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:12,682 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:12,682 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:12,682 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,682 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:12,684 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,684 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:12,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,684 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:12,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:12,685 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:12,686 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,687 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,687 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:12,687 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:12,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 23:54:12,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:12,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:12,688 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:12,688 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,688 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:12,688 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:12,688 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:12,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:12,688 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:12,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:12,688 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:12,688 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,688 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,788 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:12,789 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:12,789 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:12,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:12,791 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-08 23:54:12,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:54:12,793 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:54:12,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:12,794 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901052794 2023-02-08 23:54:12,796 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 23:54:12,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-08 23:54:12,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:12,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x55c6da90 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:12,801 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@765de41, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:12,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:13,649 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035b2a6aabe8fc4330b97af4575b2c6b2b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601035b2a6aabe8fc4330b97af4575b2c6b2b 2023-02-08 23:54:13,651 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:13,668 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:13,670 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50122, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:13,685 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:13,685 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:13,685 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:54:13,688 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39086, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:54:13,712 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6f4c294417404965853005d33b74cd74 first=Optional[r3] last=Optional[r6] 2023-02-08 23:54:13,713 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6f4c294417404965853005d33b74cd74}] 2023-02-08 23:54:13,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6f4c294417404965853005d33b74cd74 for inclusion in a9e06470572e5280a9822eb7682ca46a/family1 2023-02-08 23:54:13,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 23:54:13,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:13,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:13,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6f4c294417404965853005d33b74cd74 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__qncc8tltromqgf5imuf5gg0cb817tf5urivkbbo25215k1289eutku8ls9ophrnq/family1/6f4c294417404965853005d33b74cd74 2023-02-08 23:54:13,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__qncc8tltromqgf5imuf5gg0cb817tf5urivkbbo25215k1289eutku8ls9ophrnq/family1/6f4c294417404965853005d33b74cd74 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/ce9caebee2bb4080944bdb766b85c058_SeqId_44_ 2023-02-08 23:54:13,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__qncc8tltromqgf5imuf5gg0cb817tf5urivkbbo25215k1289eutku8ls9ophrnq/family1/6f4c294417404965853005d33b74cd74 into a9e06470572e5280a9822eb7682ca46a/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/ce9caebee2bb4080944bdb766b85c058_SeqId_44_ - updating store file list. 2023-02-08 23:54:13,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/ce9caebee2bb4080944bdb766b85c058_SeqId_44_ into a9e06470572e5280a9822eb7682ca46a/family1 2023-02-08 23:54:13,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__qncc8tltromqgf5imuf5gg0cb817tf5urivkbbo25215k1289eutku8ls9ophrnq/family1/6f4c294417404965853005d33b74cd74 into a9e06470572e5280a9822eb7682ca46a/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/ce9caebee2bb4080944bdb766b85c058_SeqId_44_) 2023-02-08 23:54:13,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__qncc8tltromqgf5imuf5gg0cb817tf5urivkbbo25215k1289eutku8ls9ophrnq/family1/6f4c294417404965853005d33b74cd74 2023-02-08 23:54:13,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 23:54:13,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 23:54:13,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): a9e06470572e5280a9822eb7682ca46a/family1 is initiating minor compaction 2023-02-08 23:54:13,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@779073ab; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:13,734 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a9e06470572e5280a9822eb7682ca46a/family1 in testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:13,734 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/7034ffd7b6f340c6a87f9aacc9365262, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/05684d359421467f9e9d55f2d8981724, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/441c47d2b4a54e238ab08bb5c79ccddc, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c5588e2b916b4038a02370466149ff73, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/4cc9a42b52854e8f894c6295408f1925, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/de6b8448ccca438a9582f105c68fcf73, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/626bc9f38ec844e08413c373f0dd72ca, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c02605da9bab4a1fbc89078e4482b1db, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/79072536fc2d40dd803da57df73d4209, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/ce9caebee2bb4080944bdb766b85c058_SeqId_44_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp, totalSize=50.0 K 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 7034ffd7b6f340c6a87f9aacc9365262, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 05684d359421467f9e9d55f2d8981724, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 441c47d2b4a54e238ab08bb5c79ccddc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting c5588e2b916b4038a02370466149ff73, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 4cc9a42b52854e8f894c6295408f1925, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting de6b8448ccca438a9582f105c68fcf73, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 626bc9f38ec844e08413c373f0dd72ca, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting c02605da9bab4a1fbc89078e4482b1db, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 79072536fc2d40dd803da57df73d4209, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 23:54:13,735 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting ce9caebee2bb4080944bdb766b85c058_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 23:54:13,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:13,737 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:13,739 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228250fa08d806840a58687f4e44d9db493 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228250fa08d806840a58687f4e44d9db493 2023-02-08 23:54:13,740 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512310c0d6349b71f4a288369894f6d99d0f1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512310c0d6349b71f4a288369894f6d99d0f1 2023-02-08 23:54:13,742 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101bdffea5679e74f1dbe5e15983e4aa92a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101bdffea5679e74f1dbe5e15983e4aa92a 2023-02-08 23:54:13,743 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103460e180ba74d4610a89582ab6bcc27a0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103460e180ba74d4610a89582ab6bcc27a0 2023-02-08 23:54:13,743 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-08 23:54:13,743 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 23:54:13,744 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:13,744 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:13,744 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:13,744 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:13,744 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:13,744 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:13,744 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:13,744 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:13,768 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): a9e06470572e5280a9822eb7682ca46a#family1#compaction#34 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:14,194 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:54:14,194 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0bed65dc to 127.0.0.1:54774 2023-02-08 23:54:14,194 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:54:14,196 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:54:14,196 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x621f5eb2 to 127.0.0.1:54774 2023-02-08 23:54:14,196 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:54:14,196 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:54:14,196 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3fe8aafd to 127.0.0.1:54774 2023-02-08 23:54:14,196 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:54:14,602 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205842d8f5ecba54bcea74939efadf4a48c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205842d8f5ecba54bcea74939efadf4a48c 2023-02-08 23:54:14,603 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:14,610 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/f0d28353128b40f08f6ab5e07bbef558 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/f0d28353128b40f08f6ab5e07bbef558 2023-02-08 23:54:14,617 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a9e06470572e5280a9822eb7682ca46a/family1 of a9e06470572e5280a9822eb7682ca46a into f0d28353128b40f08f6ab5e07bbef558(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:14,617 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:14,618 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675900453734; duration=0sec 2023-02-08 23:54:14,618 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:14,622 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:14,622 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:14,630 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c84ca95d755347e89c0b5b7fe2ab0090 first=Optional[r1] last=Optional[r2] 2023-02-08 23:54:14,631 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c84ca95d755347e89c0b5b7fe2ab0090}] 2023-02-08 23:54:14,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c84ca95d755347e89c0b5b7fe2ab0090 for inclusion in a9e06470572e5280a9822eb7682ca46a/family1 2023-02-08 23:54:14,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-08 23:54:14,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:14,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:14,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c84ca95d755347e89c0b5b7fe2ab0090 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i5mg2gmnmbm1d3f7givf79fcqr5eho6lk0jrqc56sau6vh91ip9a0tt9rjk9ulpu/family1/c84ca95d755347e89c0b5b7fe2ab0090 2023-02-08 23:54:14,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i5mg2gmnmbm1d3f7givf79fcqr5eho6lk0jrqc56sau6vh91ip9a0tt9rjk9ulpu/family1/c84ca95d755347e89c0b5b7fe2ab0090 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/907abeb6d72048a9b81bd1f1c83af8d4_SeqId_47_ 2023-02-08 23:54:14,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i5mg2gmnmbm1d3f7givf79fcqr5eho6lk0jrqc56sau6vh91ip9a0tt9rjk9ulpu/family1/c84ca95d755347e89c0b5b7fe2ab0090 into a9e06470572e5280a9822eb7682ca46a/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/907abeb6d72048a9b81bd1f1c83af8d4_SeqId_47_ - updating store file list. 2023-02-08 23:54:14,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/907abeb6d72048a9b81bd1f1c83af8d4_SeqId_47_ into a9e06470572e5280a9822eb7682ca46a/family1 2023-02-08 23:54:14,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i5mg2gmnmbm1d3f7givf79fcqr5eho6lk0jrqc56sau6vh91ip9a0tt9rjk9ulpu/family1/c84ca95d755347e89c0b5b7fe2ab0090 into a9e06470572e5280a9822eb7682ca46a/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/907abeb6d72048a9b81bd1f1c83af8d4_SeqId_47_) 2023-02-08 23:54:14,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithWeeklyPolicy__i5mg2gmnmbm1d3f7givf79fcqr5eho6lk0jrqc56sau6vh91ip9a0tt9rjk9ulpu/family1/c84ca95d755347e89c0b5b7fe2ab0090 2023-02-08 23:54:14,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 23:54:14,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 23:54:14,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): a9e06470572e5280a9822eb7682ca46a/family1 is initiating minor compaction (all files) 2023-02-08 23:54:14,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6024f4ec; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:14,657 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a9e06470572e5280a9822eb7682ca46a/family1 in testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:54:14,659 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/d399625b70ff442f85060ec0ad29c653, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/f0d28353128b40f08f6ab5e07bbef558, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/907abeb6d72048a9b81bd1f1c83af8d4_SeqId_47_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp, totalSize=176.8 K 2023-02-08 23:54:14,662 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/7034ffd7b6f340c6a87f9aacc9365262, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/05684d359421467f9e9d55f2d8981724, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/441c47d2b4a54e238ab08bb5c79ccddc, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c5588e2b916b4038a02370466149ff73, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/4cc9a42b52854e8f894c6295408f1925, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/de6b8448ccca438a9582f105c68fcf73, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/626bc9f38ec844e08413c373f0dd72ca, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c02605da9bab4a1fbc89078e4482b1db, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/79072536fc2d40dd803da57df73d4209, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/ce9caebee2bb4080944bdb766b85c058_SeqId_44_] to archive 2023-02-08 23:54:14,662 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting d399625b70ff442f85060ec0ad29c653, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838754 2023-02-08 23:54:14,663 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting f0d28353128b40f08f6ab5e07bbef558, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238754 2023-02-08 23:54:14,664 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:14,664 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 907abeb6d72048a9b81bd1f1c83af8d4_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965238754 2023-02-08 23:54:14,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:14,665 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:14,666 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/7034ffd7b6f340c6a87f9aacc9365262 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/7034ffd7b6f340c6a87f9aacc9365262 2023-02-08 23:54:14,667 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201bfc29d53eeed41cbb6c0794829bce8df to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201bfc29d53eeed41cbb6c0794829bce8df 2023-02-08 23:54:14,668 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/05684d359421467f9e9d55f2d8981724 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/05684d359421467f9e9d55f2d8981724 2023-02-08 23:54:14,668 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512054b05c1c0228b4ed99dbdd04e9673d845 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512054b05c1c0228b4ed99dbdd04e9673d845 2023-02-08 23:54:14,668 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-08 23:54:14,668 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 23:54:14,669 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:14,669 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/441c47d2b4a54e238ab08bb5c79ccddc to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/441c47d2b4a54e238ab08bb5c79ccddc 2023-02-08 23:54:14,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:14,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:14,671 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:54:14,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c5588e2b916b4038a02370466149ff73 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c5588e2b916b4038a02370466149ff73 2023-02-08 23:54:14,673 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8790 sec 2023-02-08 23:54:14,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/4cc9a42b52854e8f894c6295408f1925 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/4cc9a42b52854e8f894c6295408f1925 2023-02-08 23:54:14,676 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/de6b8448ccca438a9582f105c68fcf73 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/de6b8448ccca438a9582f105c68fcf73 2023-02-08 23:54:14,677 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/626bc9f38ec844e08413c373f0dd72ca to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/626bc9f38ec844e08413c373f0dd72ca 2023-02-08 23:54:14,679 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c02605da9bab4a1fbc89078e4482b1db to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/c02605da9bab4a1fbc89078e4482b1db 2023-02-08 23:54:14,680 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/79072536fc2d40dd803da57df73d4209 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/79072536fc2d40dd803da57df73d4209 2023-02-08 23:54:14,682 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/ce9caebee2bb4080944bdb766b85c058_SeqId_44_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/ce9caebee2bb4080944bdb766b85c058_SeqId_44_ 2023-02-08 23:54:14,687 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): a9e06470572e5280a9822eb7682ca46a#family1#compaction#35 average throughput is 12.89 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:14,784 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=252 (was 253), OpenFileDescriptor=588 (was 600), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=264 (was 286), ProcessCount=168 (was 168), AvailableMemoryMB=3390 (was 3666) 2023-02-08 23:54:14,800 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=252, OpenFileDescriptor=588, MaxFileDescriptor=60000, SystemLoadAverage=264, ProcessCount=168, AvailableMemoryMB=3389 2023-02-08 23:54:14,801 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:14,801 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:14,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 23:54:14,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:14,807 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:14,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-08 23:54:14,807 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:14,807 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:14,808 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:14,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 23:54:14,811 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:14,811 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:14,811 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:14,812 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da empty. 2023-02-08 23:54:14,812 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e empty. 2023-02-08 23:54:14,812 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1 empty. 2023-02-08 23:54:14,813 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:14,813 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:14,813 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:14,813 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-08 23:54:14,827 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:14,829 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => 6ef0d93c124a8d762314a9d7b3e5e1da, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:54:14,829 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => c091f7407da93dea8dbedd1c557bdac1, NAME => 'testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:54:14,830 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => b22d2ff0c5597d74f67d046e7f16564e, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:54:14,869 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:14,869 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing 6ef0d93c124a8d762314a9d7b3e5e1da, disabling compactions & flushes 2023-02-08 23:54:14,869 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:14,869 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:14,869 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. after waiting 0 ms 2023-02-08 23:54:14,869 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:14,869 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:14,869 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for 6ef0d93c124a8d762314a9d7b3e5e1da: Waiting for close lock at 1675900454869Disabling compacts and flushes for region at 1675900454869Disabling writes for close at 1675900454869Writing region close event to WAL at 1675900454869Closed at 1675900454869 2023-02-08 23:54:14,908 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:14,909 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:14,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 23:54:15,110 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,110 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 23:54:15,253 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:15,253 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing b22d2ff0c5597d74f67d046e7f16564e, disabling compactions & flushes 2023-02-08 23:54:15,253 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:15,253 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:15,254 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. after waiting 0 ms 2023-02-08 23:54:15,254 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:15,254 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:15,254 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for b22d2ff0c5597d74f67d046e7f16564e: Waiting for close lock at 1675900455253Disabling compacts and flushes for region at 1675900455253Disabling writes for close at 1675900455254 (+1 ms)Writing region close event to WAL at 1675900455254Closed at 1675900455254 2023-02-08 23:54:15,259 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:15,259 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing c091f7407da93dea8dbedd1c557bdac1, disabling compactions & flushes 2023-02-08 23:54:15,259 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:15,259 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:15,259 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. after waiting 0 ms 2023-02-08 23:54:15,259 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:15,259 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:15,260 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for c091f7407da93dea8dbedd1c557bdac1: Waiting for close lock at 1675900455259Disabling compacts and flushes for region at 1675900455259Disabling writes for close at 1675900455259Writing region close event to WAL at 1675900455259Closed at 1675900455259 2023-02-08 23:54:15,262 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:15,263 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900455263"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900455263"}]},"ts":"1675900455263"} 2023-02-08 23:54:15,263 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900455263"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900455263"}]},"ts":"1675900455263"} 2023-02-08 23:54:15,263 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900455263"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900455263"}]},"ts":"1675900455263"} 2023-02-08 23:54:15,266 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 23:54:15,267 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:15,267 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900455267"}]},"ts":"1675900455267"} 2023-02-08 23:54:15,268 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-08 23:54:15,271 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c091f7407da93dea8dbedd1c557bdac1, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6ef0d93c124a8d762314a9d7b3e5e1da, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b22d2ff0c5597d74f67d046e7f16564e, ASSIGN}] 2023-02-08 23:54:15,273 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b22d2ff0c5597d74f67d046e7f16564e, ASSIGN 2023-02-08 23:54:15,273 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6ef0d93c124a8d762314a9d7b3e5e1da, ASSIGN 2023-02-08 23:54:15,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c091f7407da93dea8dbedd1c557bdac1, ASSIGN 2023-02-08 23:54:15,274 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b22d2ff0c5597d74f67d046e7f16564e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:54:15,274 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c091f7407da93dea8dbedd1c557bdac1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:54:15,274 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6ef0d93c124a8d762314a9d7b3e5e1da, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:54:15,411 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,411 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 23:54:15,426 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=c091f7407da93dea8dbedd1c557bdac1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:15,426 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=b22d2ff0c5597d74f67d046e7f16564e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:15,426 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6ef0d93c124a8d762314a9d7b3e5e1da, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:15,426 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900455425"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900455425"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900455425"}]},"ts":"1675900455425"} 2023-02-08 23:54:15,426 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900455425"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900455425"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900455425"}]},"ts":"1675900455425"} 2023-02-08 23:54:15,426 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900455425"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900455425"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900455425"}]},"ts":"1675900455425"} 2023-02-08 23:54:15,428 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure c091f7407da93dea8dbedd1c557bdac1, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:15,428 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure b22d2ff0c5597d74f67d046e7f16564e, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:15,429 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 6ef0d93c124a8d762314a9d7b3e5e1da, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:15,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:15,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c091f7407da93dea8dbedd1c557bdac1, NAME => 'testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 23:54:15,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:15,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:15,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:15,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:15,585 INFO [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:15,587 DEBUG [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1/family1 2023-02-08 23:54:15,587 DEBUG [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1/family1 2023-02-08 23:54:15,587 INFO [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c091f7407da93dea8dbedd1c557bdac1 columnFamilyName family1 2023-02-08 23:54:15,589 INFO [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] regionserver.HStore(310): Store=c091f7407da93dea8dbedd1c557bdac1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:15,589 INFO [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:15,591 DEBUG [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1/family2 2023-02-08 23:54:15,591 DEBUG [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1/family2 2023-02-08 23:54:15,591 INFO [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c091f7407da93dea8dbedd1c557bdac1 columnFamilyName family2 2023-02-08 23:54:15,593 INFO [StoreOpener-c091f7407da93dea8dbedd1c557bdac1-1] regionserver.HStore(310): Store=c091f7407da93dea8dbedd1c557bdac1/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:15,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:15,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:15,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:15,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:54:15,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:15,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c091f7407da93dea8dbedd1c557bdac1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64079636, jitterRate=-0.04513901472091675}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:15,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c091f7407da93dea8dbedd1c557bdac1: Running coprocessor pre-open hook at 1675900455583Writing region info on filesystem at 1675900455583Initializing all the Stores at 1675900455584 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900455584Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900455584Cleaning up temporary data from old regions at 1675900455595 (+11 ms)Cleaning up detritus from prior splits at 1675900455595Running coprocessor post-open hooks at 1675900455600 (+5 ms)Region opened successfully at 1675900455600 2023-02-08 23:54:15,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1., pid=31, masterSystemTime=1675900455579 2023-02-08 23:54:15,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:15,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:15,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:15,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b22d2ff0c5597d74f67d046e7f16564e, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 23:54:15,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:15,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:15,604 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=c091f7407da93dea8dbedd1c557bdac1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:15,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:15,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:15,604 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900455604"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900455604"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900455604"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900455604"}]},"ts":"1675900455604"} 2023-02-08 23:54:15,605 INFO [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:15,607 DEBUG [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1 2023-02-08 23:54:15,607 DEBUG [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1 2023-02-08 23:54:15,608 INFO [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b22d2ff0c5597d74f67d046e7f16564e columnFamilyName family1 2023-02-08 23:54:15,608 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-08 23:54:15,608 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure c091f7407da93dea8dbedd1c557bdac1, server=jenkins-hbase4.apache.org,42703,1675900414768 in 178 msec 2023-02-08 23:54:15,609 INFO [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] regionserver.HStore(310): Store=b22d2ff0c5597d74f67d046e7f16564e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:15,609 INFO [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:15,610 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c091f7407da93dea8dbedd1c557bdac1, ASSIGN in 337 msec 2023-02-08 23:54:15,611 DEBUG [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family2 2023-02-08 23:54:15,611 DEBUG [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family2 2023-02-08 23:54:15,612 INFO [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b22d2ff0c5597d74f67d046e7f16564e columnFamilyName family2 2023-02-08 23:54:15,614 INFO [StoreOpener-b22d2ff0c5597d74f67d046e7f16564e-1] regionserver.HStore(310): Store=b22d2ff0c5597d74f67d046e7f16564e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:15,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:15,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:15,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:15,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:54:15,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:15,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b22d2ff0c5597d74f67d046e7f16564e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71593893, jitterRate=0.06683214008808136}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:15,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b22d2ff0c5597d74f67d046e7f16564e: Running coprocessor pre-open hook at 1675900455604Writing region info on filesystem at 1675900455604Initializing all the Stores at 1675900455605 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900455605Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900455605Cleaning up temporary data from old regions at 1675900455616 (+11 ms)Cleaning up detritus from prior splits at 1675900455617 (+1 ms)Running coprocessor post-open hooks at 1675900455622 (+5 ms)Region opened successfully at 1675900455622 2023-02-08 23:54:15,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e., pid=32, masterSystemTime=1675900455579 2023-02-08 23:54:15,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:15,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:15,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:15,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6ef0d93c124a8d762314a9d7b3e5e1da, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 23:54:15,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:15,625 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=b22d2ff0c5597d74f67d046e7f16564e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:15,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:15,625 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675900455625"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900455625"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900455625"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900455625"}]},"ts":"1675900455625"} 2023-02-08 23:54:15,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:15,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:15,627 INFO [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:15,629 DEBUG [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da/family1 2023-02-08 23:54:15,629 DEBUG [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da/family1 2023-02-08 23:54:15,629 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-02-08 23:54:15,629 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure b22d2ff0c5597d74f67d046e7f16564e, server=jenkins-hbase4.apache.org,42703,1675900414768 in 199 msec 2023-02-08 23:54:15,630 INFO [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ef0d93c124a8d762314a9d7b3e5e1da columnFamilyName family1 2023-02-08 23:54:15,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b22d2ff0c5597d74f67d046e7f16564e, ASSIGN in 358 msec 2023-02-08 23:54:15,631 INFO [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] regionserver.HStore(310): Store=6ef0d93c124a8d762314a9d7b3e5e1da/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:15,631 INFO [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:15,633 DEBUG [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da/family2 2023-02-08 23:54:15,633 DEBUG [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da/family2 2023-02-08 23:54:15,633 INFO [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ef0d93c124a8d762314a9d7b3e5e1da columnFamilyName family2 2023-02-08 23:54:15,634 INFO [StoreOpener-6ef0d93c124a8d762314a9d7b3e5e1da-1] regionserver.HStore(310): Store=6ef0d93c124a8d762314a9d7b3e5e1da/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:15,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:15,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:15,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:54:15,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:54:15,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:15,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6ef0d93c124a8d762314a9d7b3e5e1da; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74018548, jitterRate=0.10296231508255005}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:54:15,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6ef0d93c124a8d762314a9d7b3e5e1da: Running coprocessor pre-open hook at 1675900455626Writing region info on filesystem at 1675900455626Initializing all the Stores at 1675900455627 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900455627Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900455627Cleaning up temporary data from old regions at 1675900455636 (+9 ms)Cleaning up detritus from prior splits at 1675900455636Running coprocessor post-open hooks at 1675900455641 (+5 ms)Region opened successfully at 1675900455641 2023-02-08 23:54:15,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da., pid=33, masterSystemTime=1675900455579 2023-02-08 23:54:15,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:15,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:15,645 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6ef0d93c124a8d762314a9d7b3e5e1da, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:15,645 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675900455644"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900455644"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900455644"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900455644"}]},"ts":"1675900455644"} 2023-02-08 23:54:15,648 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-02-08 23:54:15,648 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 6ef0d93c124a8d762314a9d7b3e5e1da, server=jenkins-hbase4.apache.org,42703,1675900414768 in 217 msec 2023-02-08 23:54:15,650 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=27 2023-02-08 23:54:15,650 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6ef0d93c124a8d762314a9d7b3e5e1da, ASSIGN in 377 msec 2023-02-08 23:54:15,651 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:15,651 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900455651"}]},"ts":"1675900455651"} 2023-02-08 23:54:15,652 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-08 23:54:15,654 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:15,656 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 851 msec 2023-02-08 23:54:15,907 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/.tmp/family1/773bbc5350a54b49b17f6be4207a4419 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/773bbc5350a54b49b17f6be4207a4419 2023-02-08 23:54:15,912 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a9e06470572e5280a9822eb7682ca46a/family1 of a9e06470572e5280a9822eb7682ca46a into 773bbc5350a54b49b17f6be4207a4419(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:54:15,912 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a9e06470572e5280a9822eb7682ca46a: 2023-02-08 23:54:15,913 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,913 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,913 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1675900454655; duration=1sec 2023-02-08 23:54:15,913 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:15,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 23:54:15,914 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-08 23:54:15,920 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:15,920 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:15,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:15,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:15,922 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-08 23:54:15,922 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:15,922 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-08 23:54:15,922 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-08 23:54:15,923 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:15,923 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:15,924 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:15,924 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:15,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:15,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:15,924 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:15,924 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:15,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:15,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:15,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:15,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:15,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:15,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:15,925 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-08 23:54:15,925 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:15,925 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-08 23:54:15,925 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:15,925 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. started... 2023-02-08 23:54:15,925 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:54:15,926 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:15,926 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b22d2ff0c5597d74f67d046e7f16564e 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-08 23:54:15,926 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:15,925 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:15,926 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. started... 2023-02-08 23:54:15,926 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. started... 2023-02-08 23:54:15,927 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 6ef0d93c124a8d762314a9d7b3e5e1da: 2023-02-08 23:54:15,927 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for c091f7407da93dea8dbedd1c557bdac1: 2023-02-08 23:54:15,927 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:15,927 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:54:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:54:15,965 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/d399625b70ff442f85060ec0ad29c653, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/f0d28353128b40f08f6ab5e07bbef558, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/907abeb6d72048a9b81bd1f1c83af8d4_SeqId_47_] to archive 2023-02-08 23:54:15,966 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:15,968 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/d399625b70ff442f85060ec0ad29c653 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/d399625b70ff442f85060ec0ad29c653 2023-02-08 23:54:15,970 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/f0d28353128b40f08f6ab5e07bbef558 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/f0d28353128b40f08f6ab5e07bbef558 2023-02-08 23:54:15,971 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/907abeb6d72048a9b81bd1f1c83af8d4_SeqId_47_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/family1/907abeb6d72048a9b81bd1f1c83af8d4_SeqId_47_ 2023-02-08 23:54:16,348 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020824f673fdc37b40c49bbd7dcb495ed576 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020824f673fdc37b40c49bbd7dcb495ed576 2023-02-08 23:54:16,755 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/.tmp/family1/0ad45f7615b74fe7ac0ee22db1fd6bf0 2023-02-08 23:54:16,761 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/.tmp/family1/0ad45f7615b74fe7ac0ee22db1fd6bf0 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/0ad45f7615b74fe7ac0ee22db1fd6bf0 2023-02-08 23:54:16,766 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/0ad45f7615b74fe7ac0ee22db1fd6bf0, entries=1, sequenceid=5, filesize=4.9 K 2023-02-08 23:54:16,767 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for b22d2ff0c5597d74f67d046e7f16564e in 841ms, sequenceid=5, compaction requested=false 2023-02-08 23:54:16,767 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-08 23:54:16,767 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b22d2ff0c5597d74f67d046e7f16564e: 2023-02-08 23:54:16,767 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:16,768 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:54:16,768 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:54:16,768 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,768 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-08 23:54:16,768 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-08 23:54:16,771 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,771 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,772 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,772 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:16,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-08 23:54:16,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3809f1e5[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:16,774 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-08 23:54:16,774 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,775 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,775 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-08 23:54:16,775 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-08 23:54:16,775 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:16,775 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,775 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:16,776 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-08 23:54:16,777 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,777 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,777 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-08 23:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:16,781 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-08 23:54:16,781 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-08 23:54:16,781 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:16,781 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:16,781 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,781 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:16,783 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,783 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,783 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:16,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:16,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,785 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:16,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,787 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:16,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,793 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,793 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:16,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,793 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-08 23:54:16,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,793 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,793 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-08 23:54:16,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:16,793 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:16,794 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:16,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:16,794 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,794 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:16,794 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:16,794 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,794 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,794 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,794 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,894 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:16,894 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:16,894 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:16,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:16,898 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:16,899 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:16,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:16,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:16,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,901 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-08 23:54:16,901 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,901 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-08 23:54:16,901 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-08 23:54:16,901 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,901 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,903 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,903 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:16,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:16,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:16,903 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:16,903 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:16,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,903 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:16,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,904 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,904 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-08 23:54:16,904 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:16,904 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-08 23:54:16,904 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:16,904 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-08 23:54:16,904 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:16,904 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. started... 2023-02-08 23:54:16,904 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:16,905 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. started... 2023-02-08 23:54:16,905 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b22d2ff0c5597d74f67d046e7f16564e 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-08 23:54:16,904 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:54:16,905 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for c091f7407da93dea8dbedd1c557bdac1: 2023-02-08 23:54:16,905 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:54:16,905 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:16,905 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:16,905 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. started... 2023-02-08 23:54:16,905 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:54:16,905 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 6ef0d93c124a8d762314a9d7b3e5e1da: 2023-02-08 23:54:16,905 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:54:16,905 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:54:17,319 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020837379871b8564ed4894cf7c374f57185 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020837379871b8564ed4894cf7c374f57185 2023-02-08 23:54:17,328 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/.tmp/family1/a2ffee4e8fdb40a0a1403cf6c755d88e 2023-02-08 23:54:17,334 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/.tmp/family1/a2ffee4e8fdb40a0a1403cf6c755d88e as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/a2ffee4e8fdb40a0a1403cf6c755d88e 2023-02-08 23:54:17,340 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/a2ffee4e8fdb40a0a1403cf6c755d88e, entries=2, sequenceid=9, filesize=5.0 K 2023-02-08 23:54:17,341 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for b22d2ff0c5597d74f67d046e7f16564e in 436ms, sequenceid=9, compaction requested=false 2023-02-08 23:54:17,342 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b22d2ff0c5597d74f67d046e7f16564e: 2023-02-08 23:54:17,342 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:54:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:54:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-08 23:54:17,343 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-08 23:54:17,344 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,344 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,345 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,345 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-08 23:54:17,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a3cf38a[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:17,347 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-08 23:54:17,347 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,348 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,348 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,348 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:17,348 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-08 23:54:17,348 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-08 23:54:17,348 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-08 23:54:17,351 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-08 23:54:17,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:17,355 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-08 23:54:17,355 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-08 23:54:17,355 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:17,355 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:17,355 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,355 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:17,357 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,357 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:17,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,357 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,357 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,358 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:17,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:17,362 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:17,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:17,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-08 23:54:17,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:17,362 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:17,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:17,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:17,463 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:17,463 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:17,463 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:17,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:17,470 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-08 23:54:17,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-08 23:54:17,473 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-08 23:54:17,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:17,474 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901057474 2023-02-08 23:54:17,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:54:17,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-08 23:54:17,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-08 23:54:17,475 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x5595f4bf to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:17,479 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7cabaf9a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:17,479 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-08 23:54:18,308 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020826963c87278c4e51bc4a6598a5e90f2f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020826963c87278c4e51bc4a6598a5e90f2f 2023-02-08 23:54:18,309 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-08 23:54:18,324 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:18,325 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50132, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:18,332 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:18,332 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:18,333 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:54:18,335 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39088, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:54:18,345 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208/family1/09795ca7ae5b45ea8024281b1828760e first=Optional[k0] last=Optional[k1] 2023-02-08 23:54:18,346 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208/family1/09795ca7ae5b45ea8024281b1828760e}] 2023-02-08 23:54:18,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208/family1/09795ca7ae5b45ea8024281b1828760e for inclusion in b22d2ff0c5597d74f67d046e7f16564e/family1 2023-02-08 23:54:18,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-08 23:54:18,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 23:54:18,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for b22d2ff0c5597d74f67d046e7f16564e: 2023-02-08 23:54:18,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208/family1/09795ca7ae5b45ea8024281b1828760e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testScannerOnBulkLoadRefHFiles__ee1r7pcbhdn3vjs8nqi156rbp4mvain84apmcsg3g4spbfegaa8hm8vj7mfssf62/family1/09795ca7ae5b45ea8024281b1828760e 2023-02-08 23:54:18,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testScannerOnBulkLoadRefHFiles__ee1r7pcbhdn3vjs8nqi156rbp4mvain84apmcsg3g4spbfegaa8hm8vj7mfssf62/family1/09795ca7ae5b45ea8024281b1828760e as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/00cfd838a4484a3783b52a8fd21b9bda_SeqId_12_ 2023-02-08 23:54:18,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testScannerOnBulkLoadRefHFiles__ee1r7pcbhdn3vjs8nqi156rbp4mvain84apmcsg3g4spbfegaa8hm8vj7mfssf62/family1/09795ca7ae5b45ea8024281b1828760e into b22d2ff0c5597d74f67d046e7f16564e/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/00cfd838a4484a3783b52a8fd21b9bda_SeqId_12_ - updating store file list. 2023-02-08 23:54:18,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/00cfd838a4484a3783b52a8fd21b9bda_SeqId_12_ into b22d2ff0c5597d74f67d046e7f16564e/family1 2023-02-08 23:54:18,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testScannerOnBulkLoadRefHFiles__ee1r7pcbhdn3vjs8nqi156rbp4mvain84apmcsg3g4spbfegaa8hm8vj7mfssf62/family1/09795ca7ae5b45ea8024281b1828760e into b22d2ff0c5597d74f67d046e7f16564e/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/00cfd838a4484a3783b52a8fd21b9bda_SeqId_12_) 2023-02-08 23:54:18,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testScannerOnBulkLoadRefHFiles__ee1r7pcbhdn3vjs8nqi156rbp4mvain84apmcsg3g4spbfegaa8hm8vj7mfssf62/family1/09795ca7ae5b45ea8024281b1828760e 2023-02-08 23:54:18,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 23:54:18,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 23:54:18,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): b22d2ff0c5597d74f67d046e7f16564e/family1 is initiating minor compaction (all files) 2023-02-08 23:54:18,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7894f7b3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:18,363 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b22d2ff0c5597d74f67d046e7f16564e/family1 in testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:54:18,363 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/0ad45f7615b74fe7ac0ee22db1fd6bf0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/a2ffee4e8fdb40a0a1403cf6c755d88e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/00cfd838a4484a3783b52a8fd21b9bda_SeqId_12_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/.tmp, totalSize=15.1 K 2023-02-08 23:54:18,364 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 0ad45f7615b74fe7ac0ee22db1fd6bf0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900455914 2023-02-08 23:54:18,364 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting a2ffee4e8fdb40a0a1403cf6c755d88e, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900455914 2023-02-08 23:54:18,364 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 00cfd838a4484a3783b52a8fd21b9bda_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900455914 2023-02-08 23:54:18,365 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-08 23:54:18,366 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:18,367 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020824f673fdc37b40c49bbd7dcb495ed576 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020824f673fdc37b40c49bbd7dcb495ed576 2023-02-08 23:54:18,369 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020837379871b8564ed4894cf7c374f57185 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023020837379871b8564ed4894cf7c374f57185 2023-02-08 23:54:18,369 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-08 23:54:18,369 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 23:54:18,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-08 23:54:18,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-08 23:54:18,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:18,371 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-08 23:54:18,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 901 msec 2023-02-08 23:54:18,380 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): b22d2ff0c5597d74f67d046e7f16564e#family1#compaction#38 average throughput is 0.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:18,411 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=257 (was 252) - Thread LEAK? -, OpenFileDescriptor=593 (was 588) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=283 (was 264) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3378 (was 3389) 2023-02-08 23:54:18,422 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=257, OpenFileDescriptor=593, MaxFileDescriptor=60000, SystemLoadAverage=283, ProcessCount=168, AvailableMemoryMB=3378 2023-02-08 23:54:18,423 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:18,423 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:18,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 23:54:18,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:18,426 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:18,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-08 23:54:18,427 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:18,427 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:18,427 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:18,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 23:54:18,428 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:54:18,429 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080 empty. 2023-02-08 23:54:18,429 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:54:18,429 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-08 23:54:18,528 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:18,528 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:18,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 23:54:18,729 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:18,729 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:18,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 23:54:18,838 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:18,839 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 777160b56e576a2f3a7f98e3a2aa2080, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:54:18,854 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:18,854 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 777160b56e576a2f3a7f98e3a2aa2080, disabling compactions & flushes 2023-02-08 23:54:18,854 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:18,854 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:18,854 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. after waiting 0 ms 2023-02-08 23:54:18,854 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:18,854 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:18,854 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 777160b56e576a2f3a7f98e3a2aa2080: Waiting for close lock at 1675900458854Disabling compacts and flushes for region at 1675900458854Disabling writes for close at 1675900458854Writing region close event to WAL at 1675900458854Closed at 1675900458854 2023-02-08 23:54:18,858 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:18,859 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900458859"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900458859"}]},"ts":"1675900458859"} 2023-02-08 23:54:18,861 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:18,862 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:18,863 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900458862"}]},"ts":"1675900458862"} 2023-02-08 23:54:18,864 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-08 23:54:18,867 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=777160b56e576a2f3a7f98e3a2aa2080, ASSIGN}] 2023-02-08 23:54:18,869 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=777160b56e576a2f3a7f98e3a2aa2080, ASSIGN 2023-02-08 23:54:18,870 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=777160b56e576a2f3a7f98e3a2aa2080, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:54:19,021 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=777160b56e576a2f3a7f98e3a2aa2080, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:19,021 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900459021"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900459021"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900459021"}]},"ts":"1675900459021"} 2023-02-08 23:54:19,023 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 777160b56e576a2f3a7f98e3a2aa2080, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:19,030 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:19,030 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:19,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 23:54:19,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:19,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 777160b56e576a2f3a7f98e3a2aa2080, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:19,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:54:19,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:19,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:54:19,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:54:19,180 INFO [StoreOpener-777160b56e576a2f3a7f98e3a2aa2080-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:54:19,181 DEBUG [StoreOpener-777160b56e576a2f3a7f98e3a2aa2080-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1 2023-02-08 23:54:19,182 DEBUG [StoreOpener-777160b56e576a2f3a7f98e3a2aa2080-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1 2023-02-08 23:54:19,182 INFO [StoreOpener-777160b56e576a2f3a7f98e3a2aa2080-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 777160b56e576a2f3a7f98e3a2aa2080 columnFamilyName family1 2023-02-08 23:54:19,183 INFO [StoreOpener-777160b56e576a2f3a7f98e3a2aa2080-1] regionserver.HStore(310): Store=777160b56e576a2f3a7f98e3a2aa2080/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:19,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:54:19,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:54:19,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:54:19,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:19,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 777160b56e576a2f3a7f98e3a2aa2080; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59915546, jitterRate=-0.10718879103660583}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:19,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 777160b56e576a2f3a7f98e3a2aa2080: Running coprocessor pre-open hook at 1675900459179Writing region info on filesystem at 1675900459179Initializing all the Stores at 1675900459180 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675900459180Cleaning up temporary data from old regions at 1675900459185 (+5 ms)Cleaning up detritus from prior splits at 1675900459185Running coprocessor post-open hooks at 1675900459189 (+4 ms)Region opened successfully at 1675900459189 2023-02-08 23:54:19,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080., pid=37, masterSystemTime=1675900459175 2023-02-08 23:54:19,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:19,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:19,193 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=777160b56e576a2f3a7f98e3a2aa2080, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:19,193 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675900459193"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900459193"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900459193"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900459193"}]},"ts":"1675900459193"} 2023-02-08 23:54:19,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-08 23:54:19,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 777160b56e576a2f3a7f98e3a2aa2080, server=jenkins-hbase4.apache.org,42703,1675900414768 in 172 msec 2023-02-08 23:54:19,198 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-08 23:54:19,198 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=777160b56e576a2f3a7f98e3a2aa2080, ASSIGN in 329 msec 2023-02-08 23:54:19,199 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:19,199 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900459199"}]},"ts":"1675900459199"} 2023-02-08 23:54:19,201 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-08 23:54:19,202 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/.tmp/family1/262855f6530e4177bc7df93ab9900f45 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/262855f6530e4177bc7df93ab9900f45 2023-02-08 23:54:19,207 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b22d2ff0c5597d74f67d046e7f16564e/family1 of b22d2ff0c5597d74f67d046e7f16564e into 262855f6530e4177bc7df93ab9900f45(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:19,207 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b22d2ff0c5597d74f67d046e7f16564e: 2023-02-08 23:54:19,207 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1675900458363; duration=0sec 2023-02-08 23:54:19,207 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:19,207 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:19,209 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 783 msec 2023-02-08 23:54:19,258 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/0ad45f7615b74fe7ac0ee22db1fd6bf0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/a2ffee4e8fdb40a0a1403cf6c755d88e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/00cfd838a4484a3783b52a8fd21b9bda_SeqId_12_] to archive 2023-02-08 23:54:19,259 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:19,261 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/0ad45f7615b74fe7ac0ee22db1fd6bf0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/0ad45f7615b74fe7ac0ee22db1fd6bf0 2023-02-08 23:54:19,262 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/a2ffee4e8fdb40a0a1403cf6c755d88e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/a2ffee4e8fdb40a0a1403cf6c755d88e 2023-02-08 23:54:19,263 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/00cfd838a4484a3783b52a8fd21b9bda_SeqId_12_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/family1/00cfd838a4484a3783b52a8fd21b9bda_SeqId_12_ 2023-02-08 23:54:19,531 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:19,531 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:19,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 23:54:19,533 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-08 23:54:19,559 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:19,559 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:19,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:19,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:19,561 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:19,562 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:19,562 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:19,562 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:19,562 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:19,562 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:19,563 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:19,563 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:19,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:19,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:19,564 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:19,564 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:19,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:19,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:19,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:19,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:19,564 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:19,565 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:19,565 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:19,565 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:19,565 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:19,565 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:19,565 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:19,565 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:19,565 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:19,566 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 23:54:19,988 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130440f3b30a3f245b5bbddfbe1a6bd6877 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130440f3b30a3f245b5bbddfbe1a6bd6877 2023-02-08 23:54:20,392 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/65ec7bf6b3cc4b878758cbe92908f48e 2023-02-08 23:54:20,397 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/65ec7bf6b3cc4b878758cbe92908f48e as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/65ec7bf6b3cc4b878758cbe92908f48e 2023-02-08 23:54:20,401 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/65ec7bf6b3cc4b878758cbe92908f48e, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-08 23:54:20,402 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 836ms, sequenceid=5, compaction requested=false 2023-02-08 23:54:20,402 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:20,403 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:20,403 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:20,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:20,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:20,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:20,403 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:20,404 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,404 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:20,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:20,405 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,405 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:20,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:20,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1340a8ea[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:20,406 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:20,406 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,407 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,407 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:20,407 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:20,407 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:20,407 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,407 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:20,413 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:20,413 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,413 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:20,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:20,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:20,413 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:20,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:20,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:20,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:20,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:20,416 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:20,416 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:20,416 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:20,416 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:20,416 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,416 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:20,418 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,418 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:20,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,418 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:20,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:20,418 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:20,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,420 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:20,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,422 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:20,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:20,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,423 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:20,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:20,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:20,423 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:20,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:20,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:20,523 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:20,524 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:20,524 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:20,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:20,528 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:20,528 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:20,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:20,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:20,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,530 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:20,530 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:20,530 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:20,530 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:20,531 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,531 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,532 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:20,532 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:20,532 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:20,532 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,533 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,533 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:20,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:20,533 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:20,533 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:20,533 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:20,534 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:20,534 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:20,534 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:20,534 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:20,534 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 23:54:20,947 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c0cf78d15b3b4d73bec7ed5b6e009824 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128c0cf78d15b3b4d73bec7ed5b6e009824 2023-02-08 23:54:21,085 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:54:21,354 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/dfdb22b99d684585b40daf570d79f91c 2023-02-08 23:54:21,360 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/dfdb22b99d684585b40daf570d79f91c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/dfdb22b99d684585b40daf570d79f91c 2023-02-08 23:54:21,364 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/dfdb22b99d684585b40daf570d79f91c, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 23:54:21,365 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 831ms, sequenceid=9, compaction requested=false 2023-02-08 23:54:21,365 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:21,365 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:21,365 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:21,365 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:21,365 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,365 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:21,365 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:21,367 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,367 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:21,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:21,368 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,368 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:21,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:21,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:21,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:21,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:21,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29be6da0[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:21,369 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:21,369 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,376 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,376 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:21,376 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:21,376 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:21,376 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,376 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:21,378 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:21,378 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,378 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:21,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,378 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:21,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:21,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:21,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:21,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:21,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:21,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:21,380 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:21,380 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:21,380 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:21,380 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,380 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:21,383 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,383 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:21,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:21,383 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:21,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:21,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:21,384 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:21,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:21,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,385 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:21,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,387 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,387 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,387 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,387 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:21,387 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,387 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,387 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,387 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:21,387 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:21,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:21,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:21,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:21,387 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:21,388 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:21,388 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:21,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:21,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:21,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:21,488 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:21,489 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:21,489 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:21,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:21,493 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:21,493 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:21,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:21,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:21,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,496 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:21,496 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:21,496 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:21,496 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:21,496 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,496 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,497 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:21,497 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:21,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,498 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:21,498 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:21,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:21,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:21,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:21,499 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:21,499 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:21,499 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:21,499 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:21,499 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:21,499 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:21,499 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:21,499 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:21,912 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201179e07cf041b48478373292bda8dcf2d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201179e07cf041b48478373292bda8dcf2d 2023-02-08 23:54:22,318 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/a0b379d587d64fe3ade61da67895eda8 2023-02-08 23:54:22,323 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/a0b379d587d64fe3ade61da67895eda8 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/a0b379d587d64fe3ade61da67895eda8 2023-02-08 23:54:22,327 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/a0b379d587d64fe3ade61da67895eda8, entries=1, sequenceid=13, filesize=4.9 K 2023-02-08 23:54:22,328 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 829ms, sequenceid=13, compaction requested=true 2023-02-08 23:54:22,328 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:22,328 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:22,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:22,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:22,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:22,328 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:22,329 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,329 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:22,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:22,330 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,330 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:22,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:22,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:22,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e4f27ab[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:22,331 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:22,331 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,332 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,332 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:22,332 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:22,332 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:22,332 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,332 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:22,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:22,333 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:22,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,334 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:22,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:22,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:22,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:22,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:22,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:22,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:22,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:22,336 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:22,336 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:22,336 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:22,336 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,336 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:22,338 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,338 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:22,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:22,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,338 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:22,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:22,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:22,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:22,339 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,340 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,342 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:22,342 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:22,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:22,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:22,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:22,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:22,344 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,344 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,344 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:22,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:22,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:22,344 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:22,344 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,344 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,344 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,344 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,344 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:22,344 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:22,445 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:22,445 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:22,445 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:22,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:22,449 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:22,449 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:22,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:22,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:22,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,451 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:22,451 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:22,451 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:22,451 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:22,452 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,452 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,453 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,453 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:22,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:22,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:22,453 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:22,453 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:22,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:22,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,454 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,454 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:22,454 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:22,454 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:22,455 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:22,455 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:22,455 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:22,455 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:22,455 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:22,455 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:22,455 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:22,867 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205b7b1df309ee849ed889b0c7181a8053a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205b7b1df309ee849ed889b0c7181a8053a 2023-02-08 23:54:23,273 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/1e7b7d996e4c4f688d645c719c20e2d1 2023-02-08 23:54:23,278 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/1e7b7d996e4c4f688d645c719c20e2d1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/1e7b7d996e4c4f688d645c719c20e2d1 2023-02-08 23:54:23,283 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/1e7b7d996e4c4f688d645c719c20e2d1, entries=1, sequenceid=17, filesize=4.9 K 2023-02-08 23:54:23,284 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 829ms, sequenceid=17, compaction requested=true 2023-02-08 23:54:23,284 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:23,284 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:23,284 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:23,284 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:23,284 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,284 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:23,285 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:23,287 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,287 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:23,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:23,287 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,288 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:23,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:23,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:23,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3323f4b[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:23,289 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:23,289 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,291 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:23,291 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,291 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:23,292 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:23,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:23,292 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:23,292 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:23,293 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:23,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:23,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:23,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:23,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:23,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:23,295 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:23,295 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:23,295 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:23,295 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,295 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:23,297 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,297 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:23,297 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:23,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:23,298 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:23,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,299 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:23,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,303 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:23,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,303 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:23,304 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:23,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:23,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:23,404 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:23,404 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,404 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:23,408 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:23,408 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:23,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:23,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:23,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,410 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:23,411 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:23,411 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:23,411 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:23,411 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,411 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,412 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:23,412 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:23,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:23,413 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:23,413 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:23,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:23,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,414 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:23,414 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:23,414 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:23,414 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:23,414 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:23,414 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:23,414 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:23,414 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:23,414 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:23,414 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:23,829 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228110f526ef61d49179c75f4fad534be9b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228110f526ef61d49179c75f4fad534be9b 2023-02-08 23:54:24,235 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/b929727c4e8b4486bbd8fd02586ab0e3 2023-02-08 23:54:24,240 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/b929727c4e8b4486bbd8fd02586ab0e3 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/b929727c4e8b4486bbd8fd02586ab0e3 2023-02-08 23:54:24,245 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/b929727c4e8b4486bbd8fd02586ab0e3, entries=1, sequenceid=21, filesize=4.9 K 2023-02-08 23:54:24,246 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 832ms, sequenceid=21, compaction requested=true 2023-02-08 23:54:24,246 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:24,246 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:24,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:24,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:24,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:24,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:24,253 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,253 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:24,254 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,254 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:24,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:24,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:24,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:24,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15a3c03f[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:24,256 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:24,256 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,257 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,257 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:24,257 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:24,257 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:24,257 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,257 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:24,259 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:24,259 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,259 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:24,259 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:24,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:24,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:24,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:24,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:24,262 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:24,262 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:24,262 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:24,262 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,262 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:24,264 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,264 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:24,264 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:24,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:24,265 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,266 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:24,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,269 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,269 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:24,269 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:24,269 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:24,270 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,269 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:24,269 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:24,270 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,270 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:24,270 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,270 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,270 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:24,270 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:24,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:24,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:24,370 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:24,370 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:24,371 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:24,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:24,375 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:24,375 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:24,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:24,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:24,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,377 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:24,377 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:24,377 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:24,377 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:24,377 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,377 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,379 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,379 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:24,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:24,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:24,379 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:24,379 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:24,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,380 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:24,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,380 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,380 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:24,380 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:24,380 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:24,380 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:24,380 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:24,380 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:24,380 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:24,380 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:24,380 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:24,381 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:24,793 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231847be685e3874965ba91ce5eafbb38c5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231847be685e3874965ba91ce5eafbb38c5 2023-02-08 23:54:25,198 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/338b6d916bf347f59eaeeadfed43c1af 2023-02-08 23:54:25,203 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/338b6d916bf347f59eaeeadfed43c1af as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/338b6d916bf347f59eaeeadfed43c1af 2023-02-08 23:54:25,207 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/338b6d916bf347f59eaeeadfed43c1af, entries=1, sequenceid=25, filesize=4.9 K 2023-02-08 23:54:25,208 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 827ms, sequenceid=25, compaction requested=true 2023-02-08 23:54:25,208 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:25,208 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:25,208 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:25,208 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:25,209 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,209 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:25,209 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:25,210 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,210 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:25,210 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,210 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:25,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:25,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:25,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:25,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f2a6c46[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:25,212 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:25,212 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,213 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,213 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:25,213 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:25,213 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:25,213 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,213 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:25,214 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:25,214 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:25,214 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,214 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:25,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:25,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:25,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:25,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:25,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:25,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:25,216 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:25,216 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:25,216 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:25,216 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,216 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:25,219 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,219 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:25,219 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:25,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:25,219 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:25,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:25,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,220 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:25,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,222 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,222 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,222 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,222 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:25,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,222 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:25,222 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:25,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:25,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:25,222 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,223 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,223 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,223 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,223 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,223 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:25,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:25,223 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:25,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:25,323 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:25,323 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,323 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:25,328 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:25,328 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:25,328 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:25,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:25,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,330 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:25,330 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,330 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:25,330 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:25,330 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,330 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,332 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,332 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:25,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:25,332 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:25,332 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,333 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:25,333 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:25,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:25,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:25,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:25,335 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:25,335 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:25,335 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:25,335 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:25,336 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:25,748 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601014fa74ef4817741859c4be71e3534a5b6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601014fa74ef4817741859c4be71e3534a5b6 2023-02-08 23:54:26,154 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/d7230878b53f419ab1c52069b07206c0 2023-02-08 23:54:26,159 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/d7230878b53f419ab1c52069b07206c0 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/d7230878b53f419ab1c52069b07206c0 2023-02-08 23:54:26,163 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/d7230878b53f419ab1c52069b07206c0, entries=1, sequenceid=29, filesize=4.9 K 2023-02-08 23:54:26,164 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 827ms, sequenceid=29, compaction requested=true 2023-02-08 23:54:26,164 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:26,164 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:26,164 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:26,164 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:26,164 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,164 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:26,164 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:26,166 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,166 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,166 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,166 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:26,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68ff7877[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:26,168 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:26,168 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,169 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,169 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,169 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:26,169 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:26,169 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:26,169 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:26,170 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,170 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:26,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,170 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,170 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:26,173 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:26,173 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:26,173 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:26,173 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,173 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:26,175 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,175 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:26,175 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:26,176 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,177 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:26,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:26,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:26,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,180 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,180 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:26,181 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:26,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:26,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:26,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:26,281 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:26,281 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,281 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:26,285 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:26,285 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:26,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:26,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:26,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,287 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:26,287 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:26,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:26,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,291 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:26,291 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:26,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:26,292 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:26,292 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:26,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:26,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:26,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:26,293 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:26,293 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:26,293 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:26,293 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:26,293 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:26,293 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:26,293 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:26,293 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:26,706 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035ad896d48d6348d298056c30df935cb3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601035ad896d48d6348d298056c30df935cb3 2023-02-08 23:54:27,112 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/578bb9a040b940f5a019e15e0a9b09ce 2023-02-08 23:54:27,117 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/578bb9a040b940f5a019e15e0a9b09ce as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/578bb9a040b940f5a019e15e0a9b09ce 2023-02-08 23:54:27,122 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/578bb9a040b940f5a019e15e0a9b09ce, entries=1, sequenceid=33, filesize=4.9 K 2023-02-08 23:54:27,123 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 830ms, sequenceid=33, compaction requested=true 2023-02-08 23:54:27,123 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:27,123 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:27,123 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:27,126 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,126 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:27,126 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,126 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7aa5da44[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:27,127 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:27,127 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,128 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:27,129 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,129 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:27,131 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:27,131 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,131 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:27,132 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:27,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:27,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:27,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:27,134 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:27,134 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:27,134 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:27,134 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,134 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:27,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:27,136 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,138 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,141 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,142 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:27,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:27,142 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:27,242 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:27,243 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:27,243 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:27,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:27,247 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:27,247 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:27,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:27,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:27,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,250 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:27,250 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,250 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:27,250 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:27,251 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,251 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,252 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,252 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:27,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:27,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:27,252 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:27,252 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:27,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,254 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:27,254 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:27,254 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:27,254 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:27,254 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:27,254 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:27,254 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:27,254 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:27,254 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:27,254 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:27,672 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075eb9eba78a8c4d84a246c838b2df8b5d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201509075eb9eba78a8c4d84a246c838b2df8b5d 2023-02-08 23:54:28,091 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/c2213e55e7cf46f79d1df84d7ff6f833 2023-02-08 23:54:28,095 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/c2213e55e7cf46f79d1df84d7ff6f833 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/c2213e55e7cf46f79d1df84d7ff6f833 2023-02-08 23:54:28,099 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/c2213e55e7cf46f79d1df84d7ff6f833, entries=1, sequenceid=37, filesize=4.9 K 2023-02-08 23:54:28,100 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 846ms, sequenceid=37, compaction requested=true 2023-02-08 23:54:28,100 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:28,101 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:28,102 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,102 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:28,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,103 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:28,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:28,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@718c5f17[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:28,104 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:28,104 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,106 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:28,106 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:28,107 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:28,107 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,107 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:28,108 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,108 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:28,108 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:28,108 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:28,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:28,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:28,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:28,110 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:28,110 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:28,110 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:28,110 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,110 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:28,111 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,111 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:28,112 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:28,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:28,112 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:28,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,113 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:28,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:28,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:28,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:28,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:28,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 23:54:28,115 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:28,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:28,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:28,115 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:28,216 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:28,216 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,216 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:28,221 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:28,221 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:28,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:28,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:28,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,224 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:28,224 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:28,224 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 23:54:28,224 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 23:54:28,224 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,224 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,227 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,227 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:28,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:28,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:28,227 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:28,227 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:28,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:28,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 23:54:28,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:28,229 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 23:54:28,229 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:28,229 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 23:54:28,232 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:28,232 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:28,232 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:28,232 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. started... 2023-02-08 23:54:28,232 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 777160b56e576a2f3a7f98e3a2aa2080 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:28,648 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120d8a507cda5d6420bbe8dba727eb4cd9d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120d8a507cda5d6420bbe8dba727eb4cd9d 2023-02-08 23:54:29,054 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/e77ed023a2944024ad33184920bdce87 2023-02-08 23:54:29,059 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/e77ed023a2944024ad33184920bdce87 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e77ed023a2944024ad33184920bdce87 2023-02-08 23:54:29,064 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e77ed023a2944024ad33184920bdce87, entries=1, sequenceid=41, filesize=4.9 K 2023-02-08 23:54:29,065 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 777160b56e576a2f3a7f98e3a2aa2080 in 833ms, sequenceid=41, compaction requested=true 2023-02-08 23:54:29,065 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:29,065 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:29,065 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:29,065 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:29,065 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,065 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 23:54:29,065 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 23:54:29,066 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,067 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:29,067 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,067 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:29,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 23:54:29,068 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 23:54:29,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d58453c[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:29,068 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,070 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,070 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 23:54:29,070 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 23:54:29,070 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:29,070 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,070 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:29,071 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 23:54:29,071 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,071 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:29,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:29,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:29,071 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 23:54:29,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:29,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:29,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:54:29,073 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 23:54:29,073 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:29,073 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:29,073 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,073 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:29,075 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,075 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:29,075 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:29,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:29,075 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:29,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,076 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:29,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:29,087 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:29,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:29,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 23:54:29,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:29,087 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:29,188 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:29,188 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:29,188 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:29,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:29,190 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-08 23:54:29,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:54:29,192 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:54:29,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:29,193 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901069193 2023-02-08 23:54:29,197 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:54:29,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-08 23:54:29,198 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:29,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x46546906 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:29,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@19c1433f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:29,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,041 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103a6ba29b08fbd4183a467de4c9ba0f1c6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103a6ba29b08fbd4183a467de4c9ba0f1c6 2023-02-08 23:54:30,042 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:30,056 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:30,057 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36016, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:30,065 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:30,065 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:30,065 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:54:30,067 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46652, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:54:30,077 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8c1e405ee4594043bf94f9b3f27ec877 first=Optional[r3] last=Optional[r6] 2023-02-08 23:54:30,078 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8c1e405ee4594043bf94f9b3f27ec877}] 2023-02-08 23:54:30,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8c1e405ee4594043bf94f9b3f27ec877 for inclusion in 777160b56e576a2f3a7f98e3a2aa2080/family1 2023-02-08 23:54:30,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 23:54:30,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:30,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:30,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/8c1e405ee4594043bf94f9b3f27ec877 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__73tl3h43n63s5baqqr4keai928vvbtop06gl64m4mfrhiuf5vr08t02vnponntfe/family1/8c1e405ee4594043bf94f9b3f27ec877 2023-02-08 23:54:30,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__73tl3h43n63s5baqqr4keai928vvbtop06gl64m4mfrhiuf5vr08t02vnponntfe/family1/8c1e405ee4594043bf94f9b3f27ec877 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e31c0ed21bf14aefa55a16789ed8a562_SeqId_44_ 2023-02-08 23:54:30,089 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__73tl3h43n63s5baqqr4keai928vvbtop06gl64m4mfrhiuf5vr08t02vnponntfe/family1/8c1e405ee4594043bf94f9b3f27ec877 into 777160b56e576a2f3a7f98e3a2aa2080/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e31c0ed21bf14aefa55a16789ed8a562_SeqId_44_ - updating store file list. 2023-02-08 23:54:30,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e31c0ed21bf14aefa55a16789ed8a562_SeqId_44_ into 777160b56e576a2f3a7f98e3a2aa2080/family1 2023-02-08 23:54:30,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__73tl3h43n63s5baqqr4keai928vvbtop06gl64m4mfrhiuf5vr08t02vnponntfe/family1/8c1e405ee4594043bf94f9b3f27ec877 into 777160b56e576a2f3a7f98e3a2aa2080/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e31c0ed21bf14aefa55a16789ed8a562_SeqId_44_) 2023-02-08 23:54:30,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__73tl3h43n63s5baqqr4keai928vvbtop06gl64m4mfrhiuf5vr08t02vnponntfe/family1/8c1e405ee4594043bf94f9b3f27ec877 2023-02-08 23:54:30,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 23:54:30,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 23:54:30,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 777160b56e576a2f3a7f98e3a2aa2080/family1 is initiating minor compaction 2023-02-08 23:54:30,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@44edd1ea; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:30,096 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 777160b56e576a2f3a7f98e3a2aa2080/family1 in testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:54:30,096 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/dfdb22b99d684585b40daf570d79f91c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/a0b379d587d64fe3ade61da67895eda8, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/1e7b7d996e4c4f688d645c719c20e2d1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/b929727c4e8b4486bbd8fd02586ab0e3, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/338b6d916bf347f59eaeeadfed43c1af, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/d7230878b53f419ab1c52069b07206c0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/578bb9a040b940f5a019e15e0a9b09ce, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/c2213e55e7cf46f79d1df84d7ff6f833, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e77ed023a2944024ad33184920bdce87, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e31c0ed21bf14aefa55a16789ed8a562_SeqId_44_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp, totalSize=50.0 K 2023-02-08 23:54:30,096 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting dfdb22b99d684585b40daf570d79f91c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 23:54:30,096 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting a0b379d587d64fe3ade61da67895eda8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 23:54:30,097 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 1e7b7d996e4c4f688d645c719c20e2d1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 23:54:30,097 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting b929727c4e8b4486bbd8fd02586ab0e3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 23:54:30,097 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 338b6d916bf347f59eaeeadfed43c1af, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 23:54:30,097 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting d7230878b53f419ab1c52069b07206c0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 23:54:30,097 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 578bb9a040b940f5a019e15e0a9b09ce, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 23:54:30,097 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting c2213e55e7cf46f79d1df84d7ff6f833, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 23:54:30,097 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting e77ed023a2944024ad33184920bdce87, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 23:54:30,097 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting e31c0ed21bf14aefa55a16789ed8a562_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 23:54:30,099 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:30,099 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:30,101 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228110f526ef61d49179c75f4fad534be9b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228110f526ef61d49179c75f4fad534be9b 2023-02-08 23:54:30,102 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231847be685e3874965ba91ce5eafbb38c5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231847be685e3874965ba91ce5eafbb38c5 2023-02-08 23:54:30,104 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601014fa74ef4817741859c4be71e3534a5b6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601014fa74ef4817741859c4be71e3534a5b6 2023-02-08 23:54:30,105 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601035ad896d48d6348d298056c30df935cb3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601035ad896d48d6348d298056c30df935cb3 2023-02-08 23:54:30,105 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-08 23:54:30,105 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 23:54:30,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,140 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 777160b56e576a2f3a7f98e3a2aa2080#family1#compaction#49 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:30,574 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057baec150caf141558d7c461a51cda484 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512057baec150caf141558d7c461a51cda484 2023-02-08 23:54:30,575 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:30,596 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:30,596 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:30,604 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/788091fb5a2c41019021f8d36b94a7ce first=Optional[r00] last=Optional[r2] 2023-02-08 23:54:30,604 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/788091fb5a2c41019021f8d36b94a7ce}] 2023-02-08 23:54:30,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/788091fb5a2c41019021f8d36b94a7ce for inclusion in 777160b56e576a2f3a7f98e3a2aa2080/family1 2023-02-08 23:54:30,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-08 23:54:30,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:30,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:30,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/788091fb5a2c41019021f8d36b94a7ce to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1re5i907ut11pgccqnslkksgjtjnjjshcdujsp9jdnbml1jl32u8hcg7v932tgu1/family1/788091fb5a2c41019021f8d36b94a7ce 2023-02-08 23:54:30,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1re5i907ut11pgccqnslkksgjtjnjjshcdujsp9jdnbml1jl32u8hcg7v932tgu1/family1/788091fb5a2c41019021f8d36b94a7ce as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/7b52dcd7acc346afafeb91c62f2ecda4_SeqId_46_ 2023-02-08 23:54:30,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1re5i907ut11pgccqnslkksgjtjnjjshcdujsp9jdnbml1jl32u8hcg7v932tgu1/family1/788091fb5a2c41019021f8d36b94a7ce into 777160b56e576a2f3a7f98e3a2aa2080/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/7b52dcd7acc346afafeb91c62f2ecda4_SeqId_46_ - updating store file list. 2023-02-08 23:54:30,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/7b52dcd7acc346afafeb91c62f2ecda4_SeqId_46_ into 777160b56e576a2f3a7f98e3a2aa2080/family1 2023-02-08 23:54:30,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1re5i907ut11pgccqnslkksgjtjnjjshcdujsp9jdnbml1jl32u8hcg7v932tgu1/family1/788091fb5a2c41019021f8d36b94a7ce into 777160b56e576a2f3a7f98e3a2aa2080/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/7b52dcd7acc346afafeb91c62f2ecda4_SeqId_46_) 2023-02-08 23:54:30,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1re5i907ut11pgccqnslkksgjtjnjjshcdujsp9jdnbml1jl32u8hcg7v932tgu1/family1/788091fb5a2c41019021f8d36b94a7ce 2023-02-08 23:54:30,621 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:30,622 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:30,624 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130440f3b30a3f245b5bbddfbe1a6bd6877 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130440f3b30a3f245b5bbddfbe1a6bd6877 2023-02-08 23:54:30,625 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201179e07cf041b48478373292bda8dcf2d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201179e07cf041b48478373292bda8dcf2d 2023-02-08 23:54:30,627 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205b7b1df309ee849ed889b0c7181a8053a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205b7b1df309ee849ed889b0c7181a8053a 2023-02-08 23:54:30,627 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-08 23:54:30,627 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 23:54:30,627 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 23:54:30,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 23:54:30,627 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:30,629 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:54:30,631 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.4380 sec 2023-02-08 23:54:30,748 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=261 (was 257) - Thread LEAK? -, OpenFileDescriptor=631 (was 593) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 283), ProcessCount=168 (was 168), AvailableMemoryMB=3294 (was 3378) 2023-02-08 23:54:30,760 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=261, OpenFileDescriptor=631, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=168, AvailableMemoryMB=3293 2023-02-08 23:54:30,760 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:30,760 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:30,762 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 23:54:30,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:30,764 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:30,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-08 23:54:30,765 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:30,765 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:30,765 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:30,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 23:54:30,766 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:30,767 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71 empty. 2023-02-08 23:54:30,767 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:30,767 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-08 23:54:30,866 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:30,866 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:30,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 23:54:30,958 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/.tmp/family1/669eb7f58938479e8a3ee5a75e4ef137 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/669eb7f58938479e8a3ee5a75e4ef137 2023-02-08 23:54:30,963 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 777160b56e576a2f3a7f98e3a2aa2080/family1 of 777160b56e576a2f3a7f98e3a2aa2080 into 669eb7f58938479e8a3ee5a75e4ef137(size=6.6 K), total size for store is 337.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:30,963 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 777160b56e576a2f3a7f98e3a2aa2080: 2023-02-08 23:54:30,963 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675900470096; duration=0sec 2023-02-08 23:54:30,963 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:31,059 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/dfdb22b99d684585b40daf570d79f91c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/a0b379d587d64fe3ade61da67895eda8, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/1e7b7d996e4c4f688d645c719c20e2d1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/b929727c4e8b4486bbd8fd02586ab0e3, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/338b6d916bf347f59eaeeadfed43c1af, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/d7230878b53f419ab1c52069b07206c0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/578bb9a040b940f5a019e15e0a9b09ce, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/c2213e55e7cf46f79d1df84d7ff6f833, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e77ed023a2944024ad33184920bdce87, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e31c0ed21bf14aefa55a16789ed8a562_SeqId_44_] to archive 2023-02-08 23:54:31,060 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:31,062 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/dfdb22b99d684585b40daf570d79f91c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/dfdb22b99d684585b40daf570d79f91c 2023-02-08 23:54:31,063 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/a0b379d587d64fe3ade61da67895eda8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/a0b379d587d64fe3ade61da67895eda8 2023-02-08 23:54:31,064 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/1e7b7d996e4c4f688d645c719c20e2d1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/1e7b7d996e4c4f688d645c719c20e2d1 2023-02-08 23:54:31,066 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/b929727c4e8b4486bbd8fd02586ab0e3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/b929727c4e8b4486bbd8fd02586ab0e3 2023-02-08 23:54:31,067 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:31,067 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:31,067 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/338b6d916bf347f59eaeeadfed43c1af to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/338b6d916bf347f59eaeeadfed43c1af 2023-02-08 23:54:31,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 23:54:31,068 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/d7230878b53f419ab1c52069b07206c0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/d7230878b53f419ab1c52069b07206c0 2023-02-08 23:54:31,070 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/578bb9a040b940f5a019e15e0a9b09ce to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/578bb9a040b940f5a019e15e0a9b09ce 2023-02-08 23:54:31,071 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/c2213e55e7cf46f79d1df84d7ff6f833 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/c2213e55e7cf46f79d1df84d7ff6f833 2023-02-08 23:54:31,072 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e77ed023a2944024ad33184920bdce87 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e77ed023a2944024ad33184920bdce87 2023-02-08 23:54:31,073 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e31c0ed21bf14aefa55a16789ed8a562_SeqId_44_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/family1/e31c0ed21bf14aefa55a16789ed8a562_SeqId_44_ 2023-02-08 23:54:31,175 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:31,176 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 857c72d9be3c2ff762195748d825aa71, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:54:31,368 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:31,368 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:31,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 23:54:31,582 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:31,582 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing 857c72d9be3c2ff762195748d825aa71, disabling compactions & flushes 2023-02-08 23:54:31,582 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:31,582 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:31,582 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. after waiting 0 ms 2023-02-08 23:54:31,582 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:31,582 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:31,582 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for 857c72d9be3c2ff762195748d825aa71: Waiting for close lock at 1675900471582Disabling compacts and flushes for region at 1675900471582Disabling writes for close at 1675900471582Writing region close event to WAL at 1675900471582Closed at 1675900471582 2023-02-08 23:54:31,585 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:31,585 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900471585"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900471585"}]},"ts":"1675900471585"} 2023-02-08 23:54:31,587 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:31,588 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:31,588 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900471588"}]},"ts":"1675900471588"} 2023-02-08 23:54:31,589 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-08 23:54:31,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, ASSIGN}] 2023-02-08 23:54:31,594 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, ASSIGN 2023-02-08 23:54:31,595 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:54:31,746 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:31,746 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900471745"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900471745"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900471745"}]},"ts":"1675900471745"} 2023-02-08 23:54:31,747 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:31,869 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:31,869 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:31,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 23:54:31,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:31,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 857c72d9be3c2ff762195748d825aa71, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:31,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:31,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:31,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:31,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:31,903 INFO [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:31,905 DEBUG [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:31,905 DEBUG [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:31,905 INFO [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 857c72d9be3c2ff762195748d825aa71 columnFamilyName family1 2023-02-08 23:54:31,906 INFO [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] regionserver.HStore(310): Store=857c72d9be3c2ff762195748d825aa71/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:31,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:31,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:31,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:31,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:31,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 857c72d9be3c2ff762195748d825aa71; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73877275, jitterRate=0.10085718333721161}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:31,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 857c72d9be3c2ff762195748d825aa71: Running coprocessor pre-open hook at 1675900471902Writing region info on filesystem at 1675900471902Initializing all the Stores at 1675900471903 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675900471903Cleaning up temporary data from old regions at 1675900471908 (+5 ms)Cleaning up detritus from prior splits at 1675900471908Running coprocessor post-open hooks at 1675900471912 (+4 ms)Region opened successfully at 1675900471912 2023-02-08 23:54:31,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71., pid=41, masterSystemTime=1675900471899 2023-02-08 23:54:31,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:31,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:31,914 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:31,915 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900471914"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900471914"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900471914"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900471914"}]},"ts":"1675900471914"} 2023-02-08 23:54:31,918 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-08 23:54:31,918 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768 in 169 msec 2023-02-08 23:54:31,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-08 23:54:31,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, ASSIGN in 326 msec 2023-02-08 23:54:31,920 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:31,920 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900471920"}]},"ts":"1675900471920"} 2023-02-08 23:54:31,921 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-08 23:54:31,923 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:31,924 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1610 sec 2023-02-08 23:54:32,870 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:32,870 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:32,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 23:54:32,871 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-08 23:54:32,890 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:32,890 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:32,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:32,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:32,893 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:32,893 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:32,893 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:32,893 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:32,893 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:32,893 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:32,894 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:32,894 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:32,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:32,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:32,895 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:32,895 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:32,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:32,895 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:32,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:32,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:32,895 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:32,895 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:32,896 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:32,896 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:32,896 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:32,896 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:32,896 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:32,896 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:32,897 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 23:54:33,314 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d9c8885bc2fc408fb3a57e8555a434b1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130d9c8885bc2fc408fb3a57e8555a434b1 2023-02-08 23:54:33,718 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/bf085c1be3954f619e4e9251100ef7e9 2023-02-08 23:54:33,724 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/bf085c1be3954f619e4e9251100ef7e9 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/bf085c1be3954f619e4e9251100ef7e9 2023-02-08 23:54:33,729 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/bf085c1be3954f619e4e9251100ef7e9, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-08 23:54:33,730 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 833ms, sequenceid=5, compaction requested=false 2023-02-08 23:54:33,730 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:33,731 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:33,731 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:33,731 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:33,731 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:33,731 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,731 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:33,731 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:33,733 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,733 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:33,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:33,733 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:33,733 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:33,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:33,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:33,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ef7a420[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:33,734 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:33,735 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,736 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,736 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:33,736 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:33,736 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:33,736 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,736 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:33,740 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:33,740 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,740 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:33,740 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:33,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:33,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:33,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:33,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:33,743 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:33,743 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:33,743 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:33,743 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,743 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:33,744 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,745 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:33,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:33,745 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:33,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:33,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:33,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:33,745 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:33,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,747 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:33,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,750 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,750 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,750 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,750 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,750 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,750 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,750 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:33,750 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:33,750 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:33,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:33,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:33,750 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 23:54:33,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:33,750 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:33,751 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:33,751 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:33,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:33,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:33,851 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:33,851 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:33,851 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:33,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:33,856 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:33,856 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:33,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:33,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:33,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,858 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:33,858 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:33,858 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:33,858 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:33,858 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,859 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,860 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:33,860 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:33,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:33,860 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:33,860 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:33,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:33,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 23:54:33,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:33,861 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:33,861 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:33,861 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:33,861 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:33,861 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:33,862 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:33,862 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:33,862 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 23:54:34,274 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128806fdca28cef47a49250b3c288bc40ae to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128806fdca28cef47a49250b3c288bc40ae 2023-02-08 23:54:34,679 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/c50e8ae19fa74b5eb794707b475d4cc1 2023-02-08 23:54:34,684 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/c50e8ae19fa74b5eb794707b475d4cc1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/c50e8ae19fa74b5eb794707b475d4cc1 2023-02-08 23:54:34,688 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/c50e8ae19fa74b5eb794707b475d4cc1, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 23:54:34,689 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 827ms, sequenceid=9, compaction requested=false 2023-02-08 23:54:34,689 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:34,689 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:34,689 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:34,689 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:34,689 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,689 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:34,689 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:34,692 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,692 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:34,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:34,692 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,692 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:34,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:34,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:34,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:34,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:34,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a542640[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:34,693 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:34,693 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,694 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,694 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:34,694 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:34,694 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:34,695 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,695 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:34,696 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,696 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:34,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:34,696 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:34,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:34,696 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:34,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:34,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:34,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:34,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:34,698 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:34,698 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:34,698 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:34,698 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:34,698 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,698 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:34,700 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,700 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:34,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:34,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,700 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:34,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:34,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:34,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:34,701 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:34,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,702 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:34,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,710 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,710 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:34,710 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,711 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:34,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:34,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:34,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:34,711 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:34,711 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:34,711 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,711 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,711 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,711 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:34,711 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,711 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:34,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:34,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:34,811 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:34,811 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:34,811 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:34,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:34,815 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:34,815 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:34,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:34,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:34,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,817 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:34,817 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:34,818 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:34,818 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:34,818 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,818 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,819 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,819 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:34,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:34,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:34,819 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:34,819 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:34,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:34,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,821 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 23:54:34,821 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:34,821 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:34,821 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:34,821 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:34,821 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:34,821 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:34,821 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:34,821 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:34,821 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:35,233 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512011c77c1b198ea4ea182792a6e94d41437 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512011c77c1b198ea4ea182792a6e94d41437 2023-02-08 23:54:35,639 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/2bcf69a77d1f48b282096a69e64646f8 2023-02-08 23:54:35,644 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/2bcf69a77d1f48b282096a69e64646f8 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2bcf69a77d1f48b282096a69e64646f8 2023-02-08 23:54:35,647 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2bcf69a77d1f48b282096a69e64646f8, entries=1, sequenceid=13, filesize=5.0 K 2023-02-08 23:54:35,648 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 827ms, sequenceid=13, compaction requested=true 2023-02-08 23:54:35,648 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:35,648 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:35,648 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:35,648 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:35,648 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,648 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:35,648 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:35,650 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,650 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:35,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:35,650 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,650 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:35,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:35,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:35,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:35,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:35,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3933a0fa[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:35,652 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:35,652 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,654 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,654 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:35,654 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:35,654 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:35,654 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,654 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:35,656 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,656 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:35,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:35,656 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:35,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:35,656 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:35,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:35,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:35,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:35,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:35,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:35,658 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:35,658 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:35,658 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:35,658 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,658 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:35,660 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,660 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,660 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:35,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:35,660 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,661 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,663 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,663 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,663 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,663 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,664 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,664 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,664 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:35,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 23:54:35,664 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:35,664 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:35,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:35,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:35,664 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:35,664 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:35,664 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:35,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:35,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:35,764 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:35,764 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:35,764 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:35,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:35,769 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:35,769 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:35,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:35,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:35,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,772 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:35,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:35,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:35,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:35,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,772 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,774 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:35,774 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:35,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:35,775 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:35,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,775 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:35,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:35,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 23:54:35,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:35,776 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:35,776 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:35,776 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:35,776 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:35,776 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:35,776 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:35,776 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:35,777 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:35,973 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 23:54:36,195 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c3b58526223243c2b7b60f959d3ead71 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205c3b58526223243c2b7b60f959d3ead71 2023-02-08 23:54:36,600 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/32745f80ded64bfe8f056f99cfc07a91 2023-02-08 23:54:36,604 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/32745f80ded64bfe8f056f99cfc07a91 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/32745f80ded64bfe8f056f99cfc07a91 2023-02-08 23:54:36,608 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/32745f80ded64bfe8f056f99cfc07a91, entries=1, sequenceid=17, filesize=5.0 K 2023-02-08 23:54:36,609 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 832ms, sequenceid=17, compaction requested=true 2023-02-08 23:54:36,609 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:36,609 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:36,610 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:36,610 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:36,610 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,610 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:36,610 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:36,611 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,611 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:36,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:36,612 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,612 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:36,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:36,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:36,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:36,613 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:36,613 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@166d6345[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:36,613 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:36,613 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,614 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,614 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:36,614 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:36,614 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:36,614 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,614 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:36,615 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:36,615 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,615 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:36,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:36,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:36,615 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:36,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:36,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:36,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:36,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:36,617 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:36,617 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:36,617 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:36,617 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:36,617 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,617 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:36,624 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,624 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:36,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,624 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:36,624 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,625 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,628 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:36,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:36,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,628 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:36,629 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:36,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:36,729 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:36,729 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:36,729 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:36,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:36,733 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:36,733 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:36,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:36,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:36,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,735 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:36,735 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:36,735 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:36,735 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:36,736 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,736 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,737 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,737 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:36,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:36,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:36,738 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:36,738 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:36,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:36,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,738 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,739 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 23:54:36,739 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:36,739 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:36,739 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:36,739 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:36,739 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:36,739 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:36,739 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:36,739 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:36,739 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:37,152 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512284ae6a8eeaace49b4af46afc67d61afa8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512284ae6a8eeaace49b4af46afc67d61afa8 2023-02-08 23:54:37,434 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:54:37,557 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/9ca63090b0b74a1c92dfcbebb15e6591 2023-02-08 23:54:37,562 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/9ca63090b0b74a1c92dfcbebb15e6591 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/9ca63090b0b74a1c92dfcbebb15e6591 2023-02-08 23:54:37,565 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/9ca63090b0b74a1c92dfcbebb15e6591, entries=1, sequenceid=21, filesize=5.0 K 2023-02-08 23:54:37,566 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 827ms, sequenceid=21, compaction requested=true 2023-02-08 23:54:37,566 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:37,566 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:37,566 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:37,566 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:37,566 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,566 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:37,566 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:37,571 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,571 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:37,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:37,571 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,571 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:37,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:37,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:37,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:37,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:37,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5578030c[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:37,572 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:37,573 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,575 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,575 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:37,575 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:37,575 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,575 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:37,575 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:37,577 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,577 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:37,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:37,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:37,577 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:37,577 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:37,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:37,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:37,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:37,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:37,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:37,579 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:37,579 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:37,579 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:37,579 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,579 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:37,581 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,581 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:37,581 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,581 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:37,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:37,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,582 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:37,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,584 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,584 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,584 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,584 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:37,585 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:37,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:37,585 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,585 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:37,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:37,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:37,585 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,585 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,585 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:37,585 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 23:54:37,585 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:37,585 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:37,685 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:37,686 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:37,686 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:37,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:37,690 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:37,690 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:37,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:37,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:37,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,692 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:37,692 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:37,692 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:37,692 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:37,693 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,693 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,694 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,695 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:37,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:37,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:37,695 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:37,695 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:37,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:37,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,695 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 23:54:37,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:37,696 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:37,696 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:37,696 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:37,696 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:37,696 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:37,696 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:37,696 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:37,696 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:38,114 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d1603c49de3c42f99d24e366fd809717 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231d1603c49de3c42f99d24e366fd809717 2023-02-08 23:54:38,120 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/4351cdec12b8401280fd3fc23f3861f4 2023-02-08 23:54:38,124 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/4351cdec12b8401280fd3fc23f3861f4 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/4351cdec12b8401280fd3fc23f3861f4 2023-02-08 23:54:38,128 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/4351cdec12b8401280fd3fc23f3861f4, entries=1, sequenceid=25, filesize=5.0 K 2023-02-08 23:54:38,128 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 432ms, sequenceid=25, compaction requested=true 2023-02-08 23:54:38,128 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:38,129 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:38,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:38,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:38,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:38,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:38,130 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,130 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:38,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:38,131 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,131 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:38,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:38,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:38,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:38,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:38,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e1549c5[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:38,132 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:38,132 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,133 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,133 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:38,133 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,133 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:38,133 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:38,133 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:38,134 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:38,135 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:38,134 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,135 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:38,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:38,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:38,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:38,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:38,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:38,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:38,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:38,137 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:38,137 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:38,137 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:38,137 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,137 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:38,139 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,139 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:38,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:38,139 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:38,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:38,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:38,140 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:38,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,141 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:38,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:38,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:38,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:38,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:38,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:38,143 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:38,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:38,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,143 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,144 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:38,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:38,144 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:38,144 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:38,244 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:38,244 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:38,244 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:38,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:38,248 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:38,248 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:38,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:38,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:38,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,250 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:38,250 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:38,250 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:38,250 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:38,251 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,251 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,252 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,252 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:38,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:38,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:38,252 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:38,252 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:38,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:38,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 23:54:38,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:38,253 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:38,253 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:38,253 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:38,254 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:38,254 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:38,254 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:38,254 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:38,254 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:38,666 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601016f610192bf5b4104b8ea8431c498baf0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601016f610192bf5b4104b8ea8431c498baf0 2023-02-08 23:54:39,071 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/d7c526ac20ac4e3982c19a55ddd12d9a 2023-02-08 23:54:39,076 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/d7c526ac20ac4e3982c19a55ddd12d9a as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d7c526ac20ac4e3982c19a55ddd12d9a 2023-02-08 23:54:39,079 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d7c526ac20ac4e3982c19a55ddd12d9a, entries=1, sequenceid=29, filesize=5.0 K 2023-02-08 23:54:39,080 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 826ms, sequenceid=29, compaction requested=true 2023-02-08 23:54:39,080 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:39,081 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:39,081 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:39,081 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:39,081 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,081 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:39,081 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:39,082 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,082 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:39,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:39,083 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,083 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:39,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:39,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:39,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:39,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:39,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a05068d[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:39,084 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:39,084 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,086 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,086 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:39,086 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:39,086 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:39,086 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,086 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:39,087 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:39,087 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:39,087 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,087 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:39,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:39,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:39,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:39,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:39,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:39,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:39,089 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:39,089 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:39,089 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:39,089 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:39,089 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,089 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:39,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:39,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:39,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:39,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:39,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:39,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:39,091 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:39,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,092 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:39,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,097 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:39,097 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,097 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,097 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:39,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:39,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:39,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:39,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:39,097 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:39,097 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 23:54:39,097 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,097 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,097 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:39,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:39,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:39,097 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,098 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:39,098 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,098 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,198 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:39,198 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:39,198 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:39,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:39,202 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:39,202 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:39,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:39,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:39,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,205 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:39,205 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:39,205 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:39,205 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:39,205 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,205 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,206 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:39,206 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:39,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:39,207 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:39,207 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,207 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,207 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 23:54:39,207 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:39,208 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:39,208 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:39,208 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:39,208 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:39,208 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:39,208 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:39,208 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:39,208 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:39,620 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103a9af21ac75fe4189bfdcf6caa4941870 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103a9af21ac75fe4189bfdcf6caa4941870 2023-02-08 23:54:40,025 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/2b9f26bb248c46d1af1abb63ea753170 2023-02-08 23:54:40,029 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/2b9f26bb248c46d1af1abb63ea753170 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2b9f26bb248c46d1af1abb63ea753170 2023-02-08 23:54:40,033 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2b9f26bb248c46d1af1abb63ea753170, entries=1, sequenceid=33, filesize=5.0 K 2023-02-08 23:54:40,034 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 826ms, sequenceid=33, compaction requested=true 2023-02-08 23:54:40,034 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:40,034 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:40,034 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:40,034 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:40,034 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,034 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:40,034 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:40,036 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:40,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:40,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78f35c3[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:40,037 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:40,037 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,042 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,042 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:40,042 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:40,042 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:40,042 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,042 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:40,043 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:40,043 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,043 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:40,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:40,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:40,043 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:40,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:40,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:40,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:40,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:40,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:40,045 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:40,045 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:40,046 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:40,046 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,046 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:40,047 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,047 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:40,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,047 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:40,047 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,048 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,050 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,050 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,050 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:40,050 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:40,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:40,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 23:54:40,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:40,050 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,050 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:40,051 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:40,050 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:40,051 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:40,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:40,050 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,051 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,051 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,151 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:40,151 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:40,151 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:40,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:40,155 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:40,155 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:40,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:40,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:40,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,157 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:40,157 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:40,157 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:40,157 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:40,158 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,158 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,160 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:40,160 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:40,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:40,160 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,161 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:40,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:40,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,161 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,161 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 23:54:40,161 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,161 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:40,161 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:40,162 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:40,162 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:40,162 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:40,162 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:40,162 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:40,162 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:40,576 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907355444a93c31462e92105ca154e5a536 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907355444a93c31462e92105ca154e5a536 2023-02-08 23:54:40,981 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/d4ab36a758cb477d8e8c030394a7a9a7 2023-02-08 23:54:40,987 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/d4ab36a758cb477d8e8c030394a7a9a7 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d4ab36a758cb477d8e8c030394a7a9a7 2023-02-08 23:54:40,990 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d4ab36a758cb477d8e8c030394a7a9a7, entries=1, sequenceid=37, filesize=5.0 K 2023-02-08 23:54:40,991 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 829ms, sequenceid=37, compaction requested=true 2023-02-08 23:54:40,991 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:40,991 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:40,991 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:40,991 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:40,992 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,992 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:40,992 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:40,993 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,993 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:40,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:40,993 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,994 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:40,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:40,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:40,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:40,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:40,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@127a265d[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:40,995 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:40,995 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,996 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,997 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,997 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:40,997 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:40,997 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:40,997 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:40,998 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:40,998 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:40,998 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,998 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:40,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:40,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:40,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:40,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:40,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:40,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:40,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:40,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:41,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:41,000 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:41,000 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:41,000 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:41,000 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,000 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:41,003 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,003 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:41,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,003 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:41,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:41,004 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:41,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:41,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:41,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:41,005 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:41,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,009 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:41,009 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,009 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:41,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:41,010 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:41,009 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:41,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:41,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:41,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 23:54:41,010 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,010 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,010 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,010 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,010 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:41,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:41,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:41,010 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,010 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:41,110 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:41,111 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:41,111 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:41,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:41,114 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:41,114 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:41,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:41,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:41,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,117 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:41,117 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:41,117 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 23:54:41,117 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 23:54:41,117 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,117 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,118 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:41,119 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:41,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:41,119 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,119 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:41,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:41,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,119 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,120 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 23:54:41,120 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,120 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 23:54:41,120 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:41,120 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 23:54:41,120 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:41,120 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:41,120 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:41,120 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. started... 2023-02-08 23:54:41,120 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 857c72d9be3c2ff762195748d825aa71 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:41,533 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511200ee10eb272be4c78be9a4a50fffe9bf3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511200ee10eb272be4c78be9a4a50fffe9bf3 2023-02-08 23:54:41,938 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/1b053d51e28e42008d51cb65ea2a74ab 2023-02-08 23:54:41,943 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/1b053d51e28e42008d51cb65ea2a74ab as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/1b053d51e28e42008d51cb65ea2a74ab 2023-02-08 23:54:41,947 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/1b053d51e28e42008d51cb65ea2a74ab, entries=1, sequenceid=41, filesize=5.0 K 2023-02-08 23:54:41,947 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 857c72d9be3c2ff762195748d825aa71 in 827ms, sequenceid=41, compaction requested=true 2023-02-08 23:54:41,948 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:41,948 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:41,948 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:41,948 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:41,948 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,948 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 23:54:41,948 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 23:54:41,950 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,950 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:41,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:41,950 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,950 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:41,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:41,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:41,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:41,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 23:54:41,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66d51eb6[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:41,951 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 23:54:41,952 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,953 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,953 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 23:54:41,953 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 23:54:41,953 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,953 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:41,953 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:41,954 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 23:54:41,954 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,954 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:41,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:41,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:41,954 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 23:54:41,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:41,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:41,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:41,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:41,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 23:54:41,956 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 23:54:41,956 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:41,956 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:41,956 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,956 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:41,957 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,957 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:41,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:41,958 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:41,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:41,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:41,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:41,958 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:41,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,959 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:41,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,962 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,962 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:41,963 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:41,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:41,963 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:41,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 23:54:41,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:41,963 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,963 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:41,963 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:41,963 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:41,963 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:41,963 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:41,963 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:41,963 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:42,063 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:42,064 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:42,064 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:42,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:42,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:42,067 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 23:54:42,068 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 23:54:42,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:42,069 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901082069 2023-02-08 23:54:42,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:54:42,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-08 23:54:42,070 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:42,071 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x57a1f62e to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:42,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@446a8f5e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:42,075 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:42,911 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103868b84b57bad4576af51234381188319 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103868b84b57bad4576af51234381188319 2023-02-08 23:54:42,912 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:42,925 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:42,926 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59142, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:42,932 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:42,932 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:42,932 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:54:42,934 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49304, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:54:42,943 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6158876b73cd41e095b229cb1c264e1a first=Optional[r3] last=Optional[r6] 2023-02-08 23:54:42,943 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6158876b73cd41e095b229cb1c264e1a}] 2023-02-08 23:54:42,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6158876b73cd41e095b229cb1c264e1a for inclusion in 857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:42,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 23:54:42,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:42,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:42,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6158876b73cd41e095b229cb1c264e1a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uuijg0il4lcq3irj7ujvqsfd92i69kcn7g630nob8d7een83k6gru7l8vpbffvr0/family1/6158876b73cd41e095b229cb1c264e1a 2023-02-08 23:54:42,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uuijg0il4lcq3irj7ujvqsfd92i69kcn7g630nob8d7een83k6gru7l8vpbffvr0/family1/6158876b73cd41e095b229cb1c264e1a as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/35b86278f19842a1969b40a12b5eb7a5_SeqId_44_ 2023-02-08 23:54:42,953 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uuijg0il4lcq3irj7ujvqsfd92i69kcn7g630nob8d7een83k6gru7l8vpbffvr0/family1/6158876b73cd41e095b229cb1c264e1a into 857c72d9be3c2ff762195748d825aa71/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/35b86278f19842a1969b40a12b5eb7a5_SeqId_44_ - updating store file list. 2023-02-08 23:54:42,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/35b86278f19842a1969b40a12b5eb7a5_SeqId_44_ into 857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:42,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uuijg0il4lcq3irj7ujvqsfd92i69kcn7g630nob8d7een83k6gru7l8vpbffvr0/family1/6158876b73cd41e095b229cb1c264e1a into 857c72d9be3c2ff762195748d825aa71/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/35b86278f19842a1969b40a12b5eb7a5_SeqId_44_) 2023-02-08 23:54:42,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uuijg0il4lcq3irj7ujvqsfd92i69kcn7g630nob8d7een83k6gru7l8vpbffvr0/family1/6158876b73cd41e095b229cb1c264e1a 2023-02-08 23:54:42,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 23:54:42,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 23:54:42,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 857c72d9be3c2ff762195748d825aa71/family1 is initiating minor compaction 2023-02-08 23:54:42,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6981f051; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:42,960 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 857c72d9be3c2ff762195748d825aa71/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:42,960 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/c50e8ae19fa74b5eb794707b475d4cc1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2bcf69a77d1f48b282096a69e64646f8, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/32745f80ded64bfe8f056f99cfc07a91, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/9ca63090b0b74a1c92dfcbebb15e6591, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/4351cdec12b8401280fd3fc23f3861f4, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d7c526ac20ac4e3982c19a55ddd12d9a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2b9f26bb248c46d1af1abb63ea753170, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d4ab36a758cb477d8e8c030394a7a9a7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/1b053d51e28e42008d51cb65ea2a74ab, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/35b86278f19842a1969b40a12b5eb7a5_SeqId_44_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp, totalSize=50.4 K 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting c50e8ae19fa74b5eb794707b475d4cc1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 2bcf69a77d1f48b282096a69e64646f8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 32745f80ded64bfe8f056f99cfc07a91, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 9ca63090b0b74a1c92dfcbebb15e6591, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 4351cdec12b8401280fd3fc23f3861f4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting d7c526ac20ac4e3982c19a55ddd12d9a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 2b9f26bb248c46d1af1abb63ea753170, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting d4ab36a758cb477d8e8c030394a7a9a7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 1b053d51e28e42008d51cb65ea2a74ab, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 23:54:42,960 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 35b86278f19842a1969b40a12b5eb7a5_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 23:54:42,962 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:42,962 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:42,964 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512284ae6a8eeaace49b4af46afc67d61afa8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512284ae6a8eeaace49b4af46afc67d61afa8 2023-02-08 23:54:42,965 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231d1603c49de3c42f99d24e366fd809717 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231d1603c49de3c42f99d24e366fd809717 2023-02-08 23:54:42,966 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601016f610192bf5b4104b8ea8431c498baf0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601016f610192bf5b4104b8ea8431c498baf0 2023-02-08 23:54:42,968 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103a9af21ac75fe4189bfdcf6caa4941870 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103a9af21ac75fe4189bfdcf6caa4941870 2023-02-08 23:54:42,968 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-08 23:54:42,968 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 23:54:42,968 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:42,968 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:42,968 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:42,968 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:42,968 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:42,968 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:42,968 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:42,968 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:42,987 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 857c72d9be3c2ff762195748d825aa71#family1#compaction#60 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:43,804 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/a5ad38f6c9eb481db2eb27933d4b52d1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/a5ad38f6c9eb481db2eb27933d4b52d1 2023-02-08 23:54:43,809 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 857c72d9be3c2ff762195748d825aa71/family1 of 857c72d9be3c2ff762195748d825aa71 into a5ad38f6c9eb481db2eb27933d4b52d1(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:54:43,809 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:43,810 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675900482959; duration=0sec 2023-02-08 23:54:43,810 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:43,811 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120500e27ce6370b4cc6af35768fec898f74 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120500e27ce6370b4cc6af35768fec898f74 2023-02-08 23:54:43,812 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:43,828 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:43,828 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:43,835 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/416a3a89896346f6b52d2d15a39e034b first=Optional[r00] last=Optional[r2] 2023-02-08 23:54:43,836 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/416a3a89896346f6b52d2d15a39e034b}] 2023-02-08 23:54:43,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/416a3a89896346f6b52d2d15a39e034b for inclusion in 857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:43,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-08 23:54:43,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:43,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:43,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/416a3a89896346f6b52d2d15a39e034b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__jvbb3eog0csejn8vu04ajlp5kakioqcq439oi92453g7pqo3fj4jv5jhk1n1llfr/family1/416a3a89896346f6b52d2d15a39e034b 2023-02-08 23:54:43,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__jvbb3eog0csejn8vu04ajlp5kakioqcq439oi92453g7pqo3fj4jv5jhk1n1llfr/family1/416a3a89896346f6b52d2d15a39e034b as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/389b691b60bc466baff8a239fad53d22_SeqId_47_ 2023-02-08 23:54:43,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__jvbb3eog0csejn8vu04ajlp5kakioqcq439oi92453g7pqo3fj4jv5jhk1n1llfr/family1/416a3a89896346f6b52d2d15a39e034b into 857c72d9be3c2ff762195748d825aa71/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/389b691b60bc466baff8a239fad53d22_SeqId_47_ - updating store file list. 2023-02-08 23:54:43,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/389b691b60bc466baff8a239fad53d22_SeqId_47_ into 857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:43,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__jvbb3eog0csejn8vu04ajlp5kakioqcq439oi92453g7pqo3fj4jv5jhk1n1llfr/family1/416a3a89896346f6b52d2d15a39e034b into 857c72d9be3c2ff762195748d825aa71/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/389b691b60bc466baff8a239fad53d22_SeqId_47_) 2023-02-08 23:54:43,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__jvbb3eog0csejn8vu04ajlp5kakioqcq439oi92453g7pqo3fj4jv5jhk1n1llfr/family1/416a3a89896346f6b52d2d15a39e034b 2023-02-08 23:54:43,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 23:54:43,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 23:54:43,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 857c72d9be3c2ff762195748d825aa71/family1 is initiating minor compaction (all files) 2023-02-08 23:54:43,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7013f54d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:43,849 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 857c72d9be3c2ff762195748d825aa71/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:43,849 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/bf085c1be3954f619e4e9251100ef7e9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/a5ad38f6c9eb481db2eb27933d4b52d1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/389b691b60bc466baff8a239fad53d22_SeqId_47_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp, totalSize=390.1 K 2023-02-08 23:54:43,850 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting bf085c1be3954f619e4e9251100ef7e9, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838754 2023-02-08 23:54:43,850 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting a5ad38f6c9eb481db2eb27933d4b52d1, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238754 2023-02-08 23:54:43,851 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 389b691b60bc466baff8a239fad53d22_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878838754 2023-02-08 23:54:43,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:43,852 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:43,861 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130d9c8885bc2fc408fb3a57e8555a434b1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130d9c8885bc2fc408fb3a57e8555a434b1 2023-02-08 23:54:43,866 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/c50e8ae19fa74b5eb794707b475d4cc1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2bcf69a77d1f48b282096a69e64646f8, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/32745f80ded64bfe8f056f99cfc07a91, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/9ca63090b0b74a1c92dfcbebb15e6591, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/4351cdec12b8401280fd3fc23f3861f4, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d7c526ac20ac4e3982c19a55ddd12d9a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2b9f26bb248c46d1af1abb63ea753170, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d4ab36a758cb477d8e8c030394a7a9a7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/1b053d51e28e42008d51cb65ea2a74ab, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/35b86278f19842a1969b40a12b5eb7a5_SeqId_44_] to archive 2023-02-08 23:54:43,867 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:43,868 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512011c77c1b198ea4ea182792a6e94d41437 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512011c77c1b198ea4ea182792a6e94d41437 2023-02-08 23:54:43,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/c50e8ae19fa74b5eb794707b475d4cc1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/c50e8ae19fa74b5eb794707b475d4cc1 2023-02-08 23:54:43,871 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205c3b58526223243c2b7b60f959d3ead71 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205c3b58526223243c2b7b60f959d3ead71 2023-02-08 23:54:43,871 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-08 23:54:43,871 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 23:54:43,871 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:43,871 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:43,872 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:43,872 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 23:54:43,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2bcf69a77d1f48b282096a69e64646f8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2bcf69a77d1f48b282096a69e64646f8 2023-02-08 23:54:43,874 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8060 sec 2023-02-08 23:54:43,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/32745f80ded64bfe8f056f99cfc07a91 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/32745f80ded64bfe8f056f99cfc07a91 2023-02-08 23:54:43,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/9ca63090b0b74a1c92dfcbebb15e6591 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/9ca63090b0b74a1c92dfcbebb15e6591 2023-02-08 23:54:43,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/4351cdec12b8401280fd3fc23f3861f4 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/4351cdec12b8401280fd3fc23f3861f4 2023-02-08 23:54:43,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d7c526ac20ac4e3982c19a55ddd12d9a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d7c526ac20ac4e3982c19a55ddd12d9a 2023-02-08 23:54:43,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2b9f26bb248c46d1af1abb63ea753170 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/2b9f26bb248c46d1af1abb63ea753170 2023-02-08 23:54:43,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d4ab36a758cb477d8e8c030394a7a9a7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/d4ab36a758cb477d8e8c030394a7a9a7 2023-02-08 23:54:43,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/1b053d51e28e42008d51cb65ea2a74ab to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/1b053d51e28e42008d51cb65ea2a74ab 2023-02-08 23:54:43,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/35b86278f19842a1969b40a12b5eb7a5_SeqId_44_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/35b86278f19842a1969b40a12b5eb7a5_SeqId_44_ 2023-02-08 23:54:43,903 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 857c72d9be3c2ff762195748d825aa71#family1#compaction#61 average throughput is 12.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:54:43,958 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:43,958 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:43,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 23:54:43,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:44,369 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-08 23:54:44,370 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:44,370 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-08 23:54:44,372 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-08 23:54:44,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, REOPEN/MOVE}] 2023-02-08 23:54:44,376 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, REOPEN/MOVE 2023-02-08 23:54:44,376 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:44,376 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900484376"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900484376"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900484376"}]},"ts":"1675900484376"} 2023-02-08 23:54:44,378 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:44,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:44,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 857c72d9be3c2ff762195748d825aa71, disabling compactions & flushes 2023-02-08 23:54:44,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:45,135 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/.tmp/family1/eee249283c4f4e8b9971f8b2a137812d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/eee249283c4f4e8b9971f8b2a137812d 2023-02-08 23:54:45,139 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 857c72d9be3c2ff762195748d825aa71/family1 of 857c72d9be3c2ff762195748d825aa71 into eee249283c4f4e8b9971f8b2a137812d(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:54:45,139 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:45,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:45,139 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1675900483849; duration=1sec 2023-02-08 23:54:45,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:45,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. after waiting 0 ms 2023-02-08 23:54:45,139 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:54:45,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:45,140 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/bf085c1be3954f619e4e9251100ef7e9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/a5ad38f6c9eb481db2eb27933d4b52d1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/389b691b60bc466baff8a239fad53d22_SeqId_47_] to archive 2023-02-08 23:54:45,141 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:45,142 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/bf085c1be3954f619e4e9251100ef7e9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/bf085c1be3954f619e4e9251100ef7e9 2023-02-08 23:54:45,143 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/a5ad38f6c9eb481db2eb27933d4b52d1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/a5ad38f6c9eb481db2eb27933d4b52d1 2023-02-08 23:54:45,144 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/389b691b60bc466baff8a239fad53d22_SeqId_47_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/389b691b60bc466baff8a239fad53d22_SeqId_47_ 2023-02-08 23:54:45,149 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-08 23:54:45,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:45,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 857c72d9be3c2ff762195748d825aa71: Waiting for close lock at 1675900484530Running coprocessor pre-close hooks at 1675900484530Disabling compacts and flushes for region at 1675900484530Disabling writes for close at 1675900485139 (+609 ms)Writing region close event to WAL at 1675900485146 (+7 ms)Running coprocessor post-close hooks at 1675900485149 (+3 ms)Closed at 1675900485150 (+1 ms) 2023-02-08 23:54:45,150 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 857c72d9be3c2ff762195748d825aa71 to self. 2023-02-08 23:54:45,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:45,151 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=CLOSED 2023-02-08 23:54:45,151 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900485151"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900485151"}]},"ts":"1675900485151"} 2023-02-08 23:54:45,154 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-08 23:54:45,154 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768 in 775 msec 2023-02-08 23:54:45,154 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=true 2023-02-08 23:54:45,305 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:45,305 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900485305"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900485305"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900485305"}]},"ts":"1675900485305"} 2023-02-08 23:54:45,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:45,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:45,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 857c72d9be3c2ff762195748d825aa71, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:45,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:45,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:45,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:45,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:45,462 INFO [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:45,463 DEBUG [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:45,463 DEBUG [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:45,463 INFO [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 857c72d9be3c2ff762195748d825aa71 columnFamilyName family1 2023-02-08 23:54:45,468 DEBUG [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/eee249283c4f4e8b9971f8b2a137812d 2023-02-08 23:54:45,468 INFO [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] regionserver.HStore(310): Store=857c72d9be3c2ff762195748d825aa71/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:45,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:45,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:45,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:45,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 857c72d9be3c2ff762195748d825aa71; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65219389, jitterRate=-0.0281553715467453}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:45,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 857c72d9be3c2ff762195748d825aa71: Running coprocessor pre-open hook at 1675900485461Writing region info on filesystem at 1675900485461Initializing all the Stores at 1675900485462 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675900485462Cleaning up temporary data from old regions at 1675900485470 (+8 ms)Cleaning up detritus from prior splits at 1675900485470Running coprocessor post-open hooks at 1675900485473 (+3 ms)Region opened successfully at 1675900485473 2023-02-08 23:54:45,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71., pid=47, masterSystemTime=1675900485458 2023-02-08 23:54:45,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:45,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:45,475 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:45,475 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900485475"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900485475"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900485475"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900485475"}]},"ts":"1675900485475"} 2023-02-08 23:54:45,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-08 23:54:45,478 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768 in 170 msec 2023-02-08 23:54:45,479 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-08 23:54:45,479 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, REOPEN/MOVE in 1.1030 sec 2023-02-08 23:54:45,480 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-08 23:54:45,481 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1070 sec 2023-02-08 23:54:45,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5210 sec 2023-02-08 23:54:45,483 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:45,483 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:45,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-08 23:54:45,483 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:45,484 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:45,484 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:45,484 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:45,485 INFO [Listener at localhost/41521] compactions.TestMobCompactor(278): alter status finished 2023-02-08 23:54:45,486 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:45,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 23:54:45,487 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 23:54:45,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:45,488 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901085488 2023-02-08 23:54:45,488 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:54:45,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-08 23:54:45,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:45,490 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x300d5bc2 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:45,494 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@208d22e2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:45,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:45,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:45,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:45,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:45,494 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:46,319 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511282d16d047f1e24e69bb3daebb94097656 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511282d16d047f1e24e69bb3daebb94097656 2023-02-08 23:54:46,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:46,332 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:46,333 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53414, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:46,338 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:46,338 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:46,339 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:54:46,340 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55056, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:54:46,348 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/76b39cf783a7402aba289658f78c7b35 first=Optional[r05] last=Optional[r8] 2023-02-08 23:54:46,348 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=51 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/76b39cf783a7402aba289658f78c7b35}] 2023-02-08 23:54:46,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/76b39cf783a7402aba289658f78c7b35 for inclusion in 857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:46,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-08 23:54:46,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:46,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 857c72d9be3c2ff762195748d825aa71: 2023-02-08 23:54:46,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/76b39cf783a7402aba289658f78c7b35 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p2pbrh9p39vthi6je0fle0f6lkkul58a5giag5iatb9u07s6t91ngspqshbjsai0/family1/76b39cf783a7402aba289658f78c7b35 2023-02-08 23:54:46,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p2pbrh9p39vthi6je0fle0f6lkkul58a5giag5iatb9u07s6t91ngspqshbjsai0/family1/76b39cf783a7402aba289658f78c7b35 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/f4556d8fea7f4b98adba656e813f8292_SeqId_53_ 2023-02-08 23:54:46,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p2pbrh9p39vthi6je0fle0f6lkkul58a5giag5iatb9u07s6t91ngspqshbjsai0/family1/76b39cf783a7402aba289658f78c7b35 into 857c72d9be3c2ff762195748d825aa71/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/f4556d8fea7f4b98adba656e813f8292_SeqId_53_ - updating store file list. 2023-02-08 23:54:46,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/f4556d8fea7f4b98adba656e813f8292_SeqId_53_ into 857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:46,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p2pbrh9p39vthi6je0fle0f6lkkul58a5giag5iatb9u07s6t91ngspqshbjsai0/family1/76b39cf783a7402aba289658f78c7b35 into 857c72d9be3c2ff762195748d825aa71/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/f4556d8fea7f4b98adba656e813f8292_SeqId_53_) 2023-02-08 23:54:46,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p2pbrh9p39vthi6je0fle0f6lkkul58a5giag5iatb9u07s6t91ngspqshbjsai0/family1/76b39cf783a7402aba289658f78c7b35 2023-02-08 23:54:46,365 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:46,365 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:54:46,367 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511200ee10eb272be4c78be9a4a50fffe9bf3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511200ee10eb272be4c78be9a4a50fffe9bf3 2023-02-08 23:54:46,368 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128806fdca28cef47a49250b3c288bc40ae to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128806fdca28cef47a49250b3c288bc40ae 2023-02-08 23:54:46,368 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-08 23:54:46,368 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 23:54:46,368 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:46,368 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:46,368 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:46,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:46,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:46,369 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 23:54:46,370 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 883 msec 2023-02-08 23:54:46,417 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:46,418 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:46,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 23:54:46,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:46,828 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-08 23:54:46,829 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-08 23:54:46,829 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-08 23:54:46,831 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-08 23:54:46,834 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, REOPEN/MOVE}] 2023-02-08 23:54:46,834 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, REOPEN/MOVE 2023-02-08 23:54:46,835 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:46,835 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900486835"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900486835"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900486835"}]},"ts":"1675900486835"} 2023-02-08 23:54:46,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:46,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:46,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 857c72d9be3c2ff762195748d825aa71, disabling compactions & flushes 2023-02-08 23:54:46,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:46,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:46,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. after waiting 0 ms 2023-02-08 23:54:46,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:46,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-08 23:54:46,995 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:46,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 857c72d9be3c2ff762195748d825aa71: Waiting for close lock at 1675900486988Running coprocessor pre-close hooks at 1675900486988Disabling compacts and flushes for region at 1675900486988Disabling writes for close at 1675900486988Writing region close event to WAL at 1675900486991 (+3 ms)Running coprocessor post-close hooks at 1675900486995 (+4 ms)Closed at 1675900486995 2023-02-08 23:54:46,995 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 857c72d9be3c2ff762195748d825aa71 to self. 2023-02-08 23:54:46,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:46,997 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=CLOSED 2023-02-08 23:54:46,997 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900486997"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900486997"}]},"ts":"1675900486997"} 2023-02-08 23:54:46,999 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-08 23:54:46,999 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768 in 162 msec 2023-02-08 23:54:46,999 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=true 2023-02-08 23:54:47,150 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:47,150 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900487150"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900487150"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900487150"}]},"ts":"1675900487150"} 2023-02-08 23:54:47,152 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:47,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:47,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 857c72d9be3c2ff762195748d825aa71, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:47,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:47,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:47,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:47,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:47,308 INFO [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:47,309 DEBUG [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:47,309 DEBUG [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1 2023-02-08 23:54:47,309 INFO [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 857c72d9be3c2ff762195748d825aa71 columnFamilyName family1 2023-02-08 23:54:47,314 DEBUG [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/eee249283c4f4e8b9971f8b2a137812d 2023-02-08 23:54:47,317 DEBUG [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/family1/f4556d8fea7f4b98adba656e813f8292_SeqId_53_ 2023-02-08 23:54:47,317 INFO [StoreOpener-857c72d9be3c2ff762195748d825aa71-1] regionserver.HStore(310): Store=857c72d9be3c2ff762195748d825aa71/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:47,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:47,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:47,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:54:47,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 857c72d9be3c2ff762195748d825aa71; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69953371, jitterRate=0.042386457324028015}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:47,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 857c72d9be3c2ff762195748d825aa71: Running coprocessor pre-open hook at 1675900487307Writing region info on filesystem at 1675900487307Initializing all the Stores at 1675900487308 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675900487308Cleaning up temporary data from old regions at 1675900487319 (+11 ms)Cleaning up detritus from prior splits at 1675900487320 (+1 ms)Running coprocessor post-open hooks at 1675900487322 (+2 ms)Region opened successfully at 1675900487322 2023-02-08 23:54:47,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71., pid=53, masterSystemTime=1675900487303 2023-02-08 23:54:47,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:47,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:54:47,325 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=857c72d9be3c2ff762195748d825aa71, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:47,325 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675900487325"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900487325"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900487325"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900487325"}]},"ts":"1675900487325"} 2023-02-08 23:54:47,328 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-08 23:54:47,328 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 857c72d9be3c2ff762195748d825aa71, server=jenkins-hbase4.apache.org,42703,1675900414768 in 174 msec 2023-02-08 23:54:47,329 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-08 23:54:47,329 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=857c72d9be3c2ff762195748d825aa71, REOPEN/MOVE in 495 msec 2023-02-08 23:54:47,331 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-08 23:54:47,331 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 498 msec 2023-02-08 23:54:47,332 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 912 msec 2023-02-08 23:54:47,333 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:47,333 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:47,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 23:54:47,333 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:47,333 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:47,334 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:47,334 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:47,335 INFO [Listener at localhost/41521] compactions.TestMobCompactor(278): alter status finished 2023-02-08 23:54:47,336 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 23:54:47,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 23:54:47,337 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 23:54:47,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:47,338 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901087338 2023-02-08 23:54:47,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:54:47,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-08 23:54:47,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:47,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x397c32b2 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:47,343 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70af9d4d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:47,343 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:47,344 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:47,344 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:47,344 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:47,344 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:47,344 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:47,344 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:47,344 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 23:54:47,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 23:54:47,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:47,345 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 23:54:47,346 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 9 msec 2023-02-08 23:54:47,394 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=249 (was 261), OpenFileDescriptor=633 (was 631) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=183 (was 254), ProcessCount=168 (was 168), AvailableMemoryMB=3294 (was 3293) - AvailableMemoryMB LEAK? - 2023-02-08 23:54:47,405 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=249, OpenFileDescriptor=633, MaxFileDescriptor=60000, SystemLoadAverage=183, ProcessCount=168, AvailableMemoryMB=3293 2023-02-08 23:54:47,405 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:47,406 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:47,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 23:54:47,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:47,409 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:54:47,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-08 23:54:47,410 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:47,410 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:47,410 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:54:47,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 23:54:47,411 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931 2023-02-08 23:54:47,412 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931 empty. 2023-02-08 23:54:47,412 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931 2023-02-08 23:54:47,412 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-08 23:54:47,511 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:47,511 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:47,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 23:54:47,712 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:47,712 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:47,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 23:54:47,821 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-08 23:54:47,822 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2f876068b9705192eb71b2d2aa908931, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:54:48,013 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:48,013 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:48,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 23:54:48,228 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:48,228 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing 2f876068b9705192eb71b2d2aa908931, disabling compactions & flushes 2023-02-08 23:54:48,228 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:48,228 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:48,228 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. after waiting 0 ms 2023-02-08 23:54:48,228 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:48,228 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:48,228 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for 2f876068b9705192eb71b2d2aa908931: Waiting for close lock at 1675900488228Disabling compacts and flushes for region at 1675900488228Disabling writes for close at 1675900488228Writing region close event to WAL at 1675900488228Closed at 1675900488228 2023-02-08 23:54:48,231 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:54:48,232 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900488232"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900488232"}]},"ts":"1675900488232"} 2023-02-08 23:54:48,233 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:54:48,234 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:54:48,234 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900488234"}]},"ts":"1675900488234"} 2023-02-08 23:54:48,235 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-08 23:54:48,237 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, ASSIGN}] 2023-02-08 23:54:48,239 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, ASSIGN 2023-02-08 23:54:48,239 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:54:48,390 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:48,390 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900488390"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900488390"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900488390"}]},"ts":"1675900488390"} 2023-02-08 23:54:48,392 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:54:48,514 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:48,514 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:48,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 23:54:48,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:48,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2f876068b9705192eb71b2d2aa908931, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:54:48,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:54:48,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:54:48,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:54:48,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:54:48,548 INFO [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:54:48,549 DEBUG [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:54:48,550 DEBUG [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:54:48,550 INFO [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f876068b9705192eb71b2d2aa908931 columnFamilyName family1 2023-02-08 23:54:48,551 INFO [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] regionserver.HStore(310): Store=2f876068b9705192eb71b2d2aa908931/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:54:48,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931 2023-02-08 23:54:48,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931 2023-02-08 23:54:48,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:54:48,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:54:48,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2f876068b9705192eb71b2d2aa908931; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63321557, jitterRate=-0.056435272097587585}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:54:48,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2f876068b9705192eb71b2d2aa908931: Running coprocessor pre-open hook at 1675900488547Writing region info on filesystem at 1675900488547Initializing all the Stores at 1675900488548 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675900488548Cleaning up temporary data from old regions at 1675900488553 (+5 ms)Cleaning up detritus from prior splits at 1675900488554 (+1 ms)Running coprocessor post-open hooks at 1675900488559 (+5 ms)Region opened successfully at 1675900488559 2023-02-08 23:54:48,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931., pid=57, masterSystemTime=1675900488543 2023-02-08 23:54:48,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:48,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:48,562 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:48,562 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900488562"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900488562"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900488562"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900488562"}]},"ts":"1675900488562"} 2023-02-08 23:54:48,565 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-08 23:54:48,565 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768 in 172 msec 2023-02-08 23:54:48,567 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-08 23:54:48,567 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, ASSIGN in 328 msec 2023-02-08 23:54:48,567 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:54:48,567 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900488567"}]},"ts":"1675900488567"} 2023-02-08 23:54:48,568 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-08 23:54:48,570 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:54:48,572 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1640 sec 2023-02-08 23:54:49,515 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:49,515 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:49,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 23:54:49,516 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-08 23:54:49,540 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:49,540 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:49,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:49,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:49,543 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:49,543 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:49,543 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:49,543 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:49,543 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:49,543 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:49,546 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:49,546 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:49,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:49,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:49,546 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:49,546 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:49,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:49,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:49,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:49,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:49,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:49,547 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:49,547 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:49,547 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:49,547 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:49,547 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:49,547 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:49,547 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:49,547 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:49,548 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 23:54:49,969 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a750985bfb52426e81b886deac2f02cc to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130a750985bfb52426e81b886deac2f02cc 2023-02-08 23:54:50,215 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x397c32b2 to 127.0.0.1:54774 2023-02-08 23:54:50,215 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:54:50,215 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:54:50,215 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x55c6da90 to 127.0.0.1:54774 2023-02-08 23:54:50,215 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:54:50,215 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:54:50,215 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5595f4bf to 127.0.0.1:54774 2023-02-08 23:54:50,215 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:54:50,217 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:54:50,217 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x300d5bc2 to 127.0.0.1:54774 2023-02-08 23:54:50,217 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:54:50,218 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:54:50,218 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x46546906 to 127.0.0.1:54774 2023-02-08 23:54:50,218 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:54:50,218 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:54:50,218 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57a1f62e to 127.0.0.1:54774 2023-02-08 23:54:50,219 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:54:50,373 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/1f45f0439135472c873a4566842b953c 2023-02-08 23:54:50,377 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/1f45f0439135472c873a4566842b953c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/1f45f0439135472c873a4566842b953c 2023-02-08 23:54:50,380 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/1f45f0439135472c873a4566842b953c, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-08 23:54:50,381 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 833ms, sequenceid=5, compaction requested=false 2023-02-08 23:54:50,381 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:50,381 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:50,381 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:50,381 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:50,381 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:50,381 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,382 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:50,382 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:50,383 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,383 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:50,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:50,383 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,383 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:50,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:50,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:50,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:50,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:50,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34b65d8f[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:50,385 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:50,385 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,386 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,386 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:50,386 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:50,386 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:50,386 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,386 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:50,389 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,389 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:50,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:50,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:50,389 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:50,389 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:50,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:50,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:50,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:50,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:50,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:50,391 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:50,391 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:50,391 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:50,391 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,391 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:50,392 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,392 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:50,392 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:50,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:50,393 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,394 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:50,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,399 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,399 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:50,400 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:50,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:50,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:50,400 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:50,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:50,400 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:50,400 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,400 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,400 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,400 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:50,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:50,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:50,400 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:50,400 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,400 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,500 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:50,500 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:50,501 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:50,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:50,504 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:50,504 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:50,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:50,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:50,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,507 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:50,507 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:50,507 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:50,507 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:50,507 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,507 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,509 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,509 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:50,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:50,509 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:50,509 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:50,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 23:54:50,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:50,510 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:50,510 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:50,510 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:50,511 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:50,511 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:50,511 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:50,511 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:50,511 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 23:54:50,721 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:54:50,922 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a47d97376ec64afea3863d5b66b8deac to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128a47d97376ec64afea3863d5b66b8deac 2023-02-08 23:54:51,328 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/80c509f63cbd4486995736daffd6776e 2023-02-08 23:54:51,332 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/80c509f63cbd4486995736daffd6776e as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/80c509f63cbd4486995736daffd6776e 2023-02-08 23:54:51,336 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/80c509f63cbd4486995736daffd6776e, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 23:54:51,336 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 825ms, sequenceid=9, compaction requested=false 2023-02-08 23:54:51,337 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:51,337 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:51,337 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:51,337 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:51,337 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,337 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:51,337 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:51,338 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,338 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:51,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:51,339 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,339 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:51,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:51,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:51,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:51,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:51,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a6947f7[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:51,340 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:51,340 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,341 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,341 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:51,341 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:51,341 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:51,341 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,341 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:51,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:51,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:51,342 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,342 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:51,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:51,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:51,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:51,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:51,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:51,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:51,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:51,344 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:51,345 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:51,345 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:51,345 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,345 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:51,347 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,347 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:51,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,347 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:51,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:51,347 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:51,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,348 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:51,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,350 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,350 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,350 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,350 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:51,350 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:51,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 23:54:51,350 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:51,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:51,350 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:51,350 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:51,350 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:51,351 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,351 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:51,351 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:51,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:51,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:51,451 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:51,451 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:51,451 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:51,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:51,455 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:51,455 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:51,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:51,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:51,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,457 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:51,457 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:51,457 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:51,457 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:51,458 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,458 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,459 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,459 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:51,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:51,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:51,459 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:51,459 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:51,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:51,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,460 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,460 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 23:54:51,460 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:51,460 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:51,460 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:51,460 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:51,460 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:51,461 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:51,461 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:51,461 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:51,461 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:51,875 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c13f0b282c6648b8af265f35f226204e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c13f0b282c6648b8af265f35f226204e 2023-02-08 23:54:52,281 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/941187bb1e034d9e9124547d1817f18d 2023-02-08 23:54:52,286 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/941187bb1e034d9e9124547d1817f18d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/941187bb1e034d9e9124547d1817f18d 2023-02-08 23:54:52,290 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/941187bb1e034d9e9124547d1817f18d, entries=1, sequenceid=13, filesize=5.0 K 2023-02-08 23:54:52,290 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 829ms, sequenceid=13, compaction requested=true 2023-02-08 23:54:52,290 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:52,290 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:52,291 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:52,291 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:52,291 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,291 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:52,291 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:52,292 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,292 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:52,292 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,292 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:52,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:52,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9e2c829[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:52,294 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:52,294 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,299 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,299 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:52,299 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:52,299 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:52,299 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,299 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:52,300 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:52,300 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:52,300 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,300 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:52,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:52,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:52,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:52,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:52,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:52,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:52,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:52,302 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:52,302 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:52,302 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:52,302 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,302 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:52,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:52,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:52,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,303 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:52,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:52,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:52,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:52,304 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:52,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,305 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:52,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,307 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:52,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:52,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:52,307 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:52,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:52,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:52,408 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:52,408 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:52,408 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:52,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:52,411 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:52,411 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:52,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:52,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:52,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,413 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:52,413 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:52,413 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:52,413 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:52,414 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,414 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,416 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:52,416 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:52,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:52,416 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:52,416 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:52,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:52,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,417 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 23:54:52,417 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:52,417 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:52,417 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:52,417 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:52,417 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:52,417 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:52,417 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:52,417 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:52,417 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:52,837 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120573f9b2be7d6e4798b0a79a04790d6843 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120573f9b2be7d6e4798b0a79a04790d6843 2023-02-08 23:54:53,243 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/a04a4d7c03604b4da24fc17aa23ecfb6 2023-02-08 23:54:53,250 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/a04a4d7c03604b4da24fc17aa23ecfb6 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a04a4d7c03604b4da24fc17aa23ecfb6 2023-02-08 23:54:53,254 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a04a4d7c03604b4da24fc17aa23ecfb6, entries=1, sequenceid=17, filesize=5.0 K 2023-02-08 23:54:53,254 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 837ms, sequenceid=17, compaction requested=true 2023-02-08 23:54:53,255 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:53,255 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:53,255 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:53,255 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:53,255 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,255 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:53,255 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:53,259 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,259 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:53,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:53,259 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,259 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:53,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:53,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:53,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:53,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@478c570d[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:53,261 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:53,261 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,262 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,262 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:53,262 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:53,262 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:53,262 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,262 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:53,264 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:53,264 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:53,264 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,264 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:53,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:53,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:53,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:53,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:53,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:53,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:53,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:53,266 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:53,266 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:53,266 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:53,266 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,266 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:53,268 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,268 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:53,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:53,268 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:53,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:53,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:53,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:53,269 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:53,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,270 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:53,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,272 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,272 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:53,272 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:53,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:53,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:53,272 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:53,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:53,272 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:53,272 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,272 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,273 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:53,273 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:53,273 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:53,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:53,273 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,273 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,373 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:53,373 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:53,373 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:53,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:53,377 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:53,377 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:53,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:53,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:53,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,379 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:53,379 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:53,379 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:53,379 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:53,379 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,380 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,381 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,381 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:53,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:53,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:53,381 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:53,381 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:53,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,381 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:53,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,382 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,382 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 23:54:53,382 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:53,382 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:53,382 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:53,382 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:53,382 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:53,382 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:53,382 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:53,382 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:53,383 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:53,795 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228f91c85e5aabe495faf1612671e4c228b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228f91c85e5aabe495faf1612671e4c228b 2023-02-08 23:54:54,200 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/983e19fcd204433882ab0a7da4bad95f 2023-02-08 23:54:54,205 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/983e19fcd204433882ab0a7da4bad95f as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/983e19fcd204433882ab0a7da4bad95f 2023-02-08 23:54:54,209 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/983e19fcd204433882ab0a7da4bad95f, entries=1, sequenceid=21, filesize=5.0 K 2023-02-08 23:54:54,210 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 828ms, sequenceid=21, compaction requested=true 2023-02-08 23:54:54,210 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:54,210 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:54,210 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:54,213 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,213 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:54,213 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,213 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:54,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:54,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:54,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:54,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:54,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41d03056[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:54,214 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:54,214 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,216 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:54,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:54,216 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,216 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:54,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:54,217 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:54,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,217 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:54,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:54,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:54,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:54,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:54,219 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:54,219 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:54,219 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:54,219 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,219 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:54,221 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,221 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,221 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:54,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:54,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:54,221 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:54,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:54,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,222 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:54,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:54,226 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:54,226 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 23:54:54,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:54,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:54,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:54,226 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:54,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:54,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:54,326 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:54,327 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:54,327 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:54,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:54,330 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:54,330 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:54,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:54,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:54,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,332 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:54,332 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:54,332 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:54,332 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:54,333 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,333 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,334 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:54,334 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:54,334 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:54,334 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,335 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 23:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:54,335 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:54,335 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:54,335 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:54,335 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:54,335 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:54,335 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:54,335 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:54,336 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:54,747 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ea0037d9f722443abef635942ffb8b34 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231ea0037d9f722443abef635942ffb8b34 2023-02-08 23:54:55,152 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/4388d1d37d834ea1b1c713fb49d8736d 2023-02-08 23:54:55,157 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/4388d1d37d834ea1b1c713fb49d8736d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/4388d1d37d834ea1b1c713fb49d8736d 2023-02-08 23:54:55,161 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/4388d1d37d834ea1b1c713fb49d8736d, entries=1, sequenceid=25, filesize=5.0 K 2023-02-08 23:54:55,162 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 826ms, sequenceid=25, compaction requested=true 2023-02-08 23:54:55,162 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:55,162 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:55,162 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:55,162 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:55,162 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,162 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:55,162 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:55,164 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,164 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:55,164 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,164 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:55,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:55,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41868164[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:55,166 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:55,166 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,167 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,167 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:55,167 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,167 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:55,167 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:55,167 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:55,169 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:55,169 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:55,169 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,169 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:55,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:55,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:55,172 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:55,172 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:55,172 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:55,172 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,172 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:55,173 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,173 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,173 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:55,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:55,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:55,174 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,175 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:55,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,179 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,179 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:55,179 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:55,180 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:55,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:55,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:55,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:55,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,179 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:55,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:55,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:55,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,180 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:55,180 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:55,180 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,280 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:55,280 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:55,280 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:55,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:55,285 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:55,285 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:55,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:55,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:55,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,287 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:55,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:55,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:55,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:55,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,288 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,292 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:55,292 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:55,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:55,292 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:55,293 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:55,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:55,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 23:54:55,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:55,293 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:55,294 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:55,294 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:55,294 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:55,706 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010110dc2a6acb4b4058a9a8d6af8cf85b02 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010110dc2a6acb4b4058a9a8d6af8cf85b02 2023-02-08 23:54:56,112 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/02a163a6b79e4dd5bf2d11bdc03e0d32 2023-02-08 23:54:56,116 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/02a163a6b79e4dd5bf2d11bdc03e0d32 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/02a163a6b79e4dd5bf2d11bdc03e0d32 2023-02-08 23:54:56,120 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/02a163a6b79e4dd5bf2d11bdc03e0d32, entries=1, sequenceid=29, filesize=5.0 K 2023-02-08 23:54:56,121 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 827ms, sequenceid=29, compaction requested=true 2023-02-08 23:54:56,121 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:56,121 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:56,121 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:56,121 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:56,121 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,121 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:56,121 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:56,123 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,123 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:56,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:56,123 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,124 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:56,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:56,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:56,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:56,125 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:56,125 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ebabafd[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:56,125 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:56,125 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,126 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,126 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:56,126 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:56,126 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:56,126 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,126 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:56,128 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:56,128 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,128 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:56,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,128 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:56,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:56,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:56,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:56,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:56,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:56,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:56,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:56,130 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:56,130 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:56,130 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:56,130 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,130 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:56,131 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,131 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:56,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:56,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,131 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:56,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:56,132 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:56,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:56,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:56,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,133 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:56,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:56,136 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 23:54:56,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:56,136 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:56,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:56,136 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:56,137 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:56,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:56,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:56,237 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:56,237 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:56,237 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:56,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:56,240 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:56,241 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:56,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:56,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:56,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,243 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:56,243 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:56,243 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:56,243 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:56,243 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,243 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,244 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:56,244 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:56,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:56,245 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:56,245 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:56,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,245 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:56,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,246 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 23:54:56,246 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:56,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:56,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:56,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:56,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:56,246 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:56,246 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:56,246 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:56,246 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:56,666 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b831623478b94abbbaeb5ace574e3b95 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103b831623478b94abbbaeb5ace574e3b95 2023-02-08 23:54:57,071 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/ecc77f4cce794a5490073c659f8497cf 2023-02-08 23:54:57,075 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/ecc77f4cce794a5490073c659f8497cf as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/ecc77f4cce794a5490073c659f8497cf 2023-02-08 23:54:57,078 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/ecc77f4cce794a5490073c659f8497cf, entries=1, sequenceid=33, filesize=5.0 K 2023-02-08 23:54:57,079 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 833ms, sequenceid=33, compaction requested=true 2023-02-08 23:54:57,079 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:57,079 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:57,079 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:57,081 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,081 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:57,081 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,081 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:57,083 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:57,083 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b2bba18[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:57,083 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:57,083 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,084 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,084 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:57,084 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:57,084 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:57,084 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,084 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:57,086 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,086 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:57,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,086 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:57,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:57,087 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:57,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:57,088 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:57,089 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:57,089 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:57,089 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,089 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:57,090 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,090 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:57,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:57,090 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:57,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:57,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:57,091 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,092 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:57,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:57,094 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:57,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:57,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:57,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:57,094 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:57,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:57,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:57,194 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:57,194 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:57,194 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:57,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:57,198 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:57,198 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:57,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:57,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:57,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,200 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:57,200 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:57,200 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:57,201 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:57,201 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,201 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,203 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:57,203 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:57,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:57,203 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:57,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,203 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:57,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:57,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,204 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 23:54:57,204 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:57,204 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:57,204 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:57,204 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:57,204 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:57,205 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:57,205 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:57,205 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:57,205 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:57,619 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907bef2f456f3bc4c1abc1bf6bbbd6d787e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907bef2f456f3bc4c1abc1bf6bbbd6d787e 2023-02-08 23:54:58,025 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/d3b754f744cc4ef293a364e5572ceab0 2023-02-08 23:54:58,029 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/d3b754f744cc4ef293a364e5572ceab0 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/d3b754f744cc4ef293a364e5572ceab0 2023-02-08 23:54:58,032 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/d3b754f744cc4ef293a364e5572ceab0, entries=1, sequenceid=37, filesize=5.0 K 2023-02-08 23:54:58,033 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 828ms, sequenceid=37, compaction requested=true 2023-02-08 23:54:58,033 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:58,033 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:58,033 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:58,033 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:58,033 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,033 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:58,034 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:58,035 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,035 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:58,035 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,035 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:58,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:58,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b3977ab[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:58,037 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:58,037 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,039 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:58,040 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,040 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:58,041 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:58,041 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:58,041 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,041 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:58,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:58,043 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:58,044 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:58,044 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:58,044 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,044 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:58,045 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,045 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:58,045 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:58,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:58,045 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,046 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:58,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,048 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,048 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:58,048 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:58,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:58,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:58,048 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,048 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:58,048 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:58,048 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:58,049 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,049 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,049 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:58,049 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:58,049 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:58,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:58,149 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:58,149 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:58,149 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:58,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:58,153 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:58,153 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:58,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:54:58,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 23:54:58,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,155 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:58,155 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:58,155 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 23:54:58,155 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 23:54:58,155 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,155 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:58,157 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:58,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:58,157 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,157 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:54:58,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,158 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:54:58,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,158 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,158 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 23:54:58,158 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,159 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 23:54:58,159 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:54:58,159 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 23:54:58,159 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:54:58,159 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:54:58,159 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:58,159 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. started... 2023-02-08 23:54:58,159 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f876068b9705192eb71b2d2aa908931 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:54:58,571 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ee3ee527c57a41f1ad72927a3df7d08f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120ee3ee527c57a41f1ad72927a3df7d08f 2023-02-08 23:54:58,975 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/a4eea1b433144d328e6d4cd32820bee6 2023-02-08 23:54:58,979 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/a4eea1b433144d328e6d4cd32820bee6 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a4eea1b433144d328e6d4cd32820bee6 2023-02-08 23:54:58,983 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a4eea1b433144d328e6d4cd32820bee6, entries=1, sequenceid=41, filesize=5.0 K 2023-02-08 23:54:58,984 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2f876068b9705192eb71b2d2aa908931 in 825ms, sequenceid=41, compaction requested=true 2023-02-08 23:54:58,984 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:58,984 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:54:58,984 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:54:58,984 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:54:58,984 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,984 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 23:54:58,984 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 23:54:58,986 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,986 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:58,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:58,986 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,986 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:54:58,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:58,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:58,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:58,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 23:54:58,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e414c09[Count = 0] remaining members to acquire global barrier 2023-02-08 23:54:58,987 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 23:54:58,987 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,988 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,988 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 23:54:58,988 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 23:54:58,988 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:54:58,988 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,989 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:54:58,990 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 23:54:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 23:54:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:58,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:54:58,992 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 23:54:58,992 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 23:54:58,992 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:54:58,992 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:54:58,992 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,992 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:54:58,995 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,995 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:54:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:54:58,995 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:54:58,995 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:54:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,996 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:54:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,998 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,998 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:54:58,998 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:54:58,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 23:54:58,998 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 23:54:58,998 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:54:58,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:54:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:54:58,998 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:54:58,998 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,999 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:54:58,999 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,999 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,999 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:58,999 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 23:54:58,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:54:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:54:58,999 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:54:59,099 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:54:59,099 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:59,099 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:54:59,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:54:59,101 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:54:59,102 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 23:54:59,103 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 23:54:59,104 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901099104 2023-02-08 23:54:59,104 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:54:59,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:54:59,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-08 23:54:59,107 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:54:59,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x1fef556f to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:54:59,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@663ed400, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:54:59,112 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:54:59,959 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103300caf685eaf4f319d9f73356c53b0b7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103300caf685eaf4f319d9f73356c53b0b7 2023-02-08 23:54:59,959 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:54:59,972 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:54:59,973 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56356, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:54:59,978 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:59,978 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:54:59,979 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:54:59,980 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48290, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:54:59,987 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ad40ac4a924c410b8939155c223b560f first=Optional[r3] last=Optional[r6] 2023-02-08 23:54:59,987 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ad40ac4a924c410b8939155c223b560f}] 2023-02-08 23:54:59,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ad40ac4a924c410b8939155c223b560f for inclusion in 2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:54:59,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 23:54:59,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:54:59,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:54:59,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ad40ac4a924c410b8939155c223b560f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vtiosgts5ed5rgg788miagjee527jjsp0u078teg0d0grhdfmbb8p5nfgsthcjcb/family1/ad40ac4a924c410b8939155c223b560f 2023-02-08 23:54:59,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vtiosgts5ed5rgg788miagjee527jjsp0u078teg0d0grhdfmbb8p5nfgsthcjcb/family1/ad40ac4a924c410b8939155c223b560f as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/8eb1fdb4bcb14ae181f035af5d12ba9d_SeqId_44_ 2023-02-08 23:54:59,996 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vtiosgts5ed5rgg788miagjee527jjsp0u078teg0d0grhdfmbb8p5nfgsthcjcb/family1/ad40ac4a924c410b8939155c223b560f into 2f876068b9705192eb71b2d2aa908931/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/8eb1fdb4bcb14ae181f035af5d12ba9d_SeqId_44_ - updating store file list. 2023-02-08 23:54:59,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/8eb1fdb4bcb14ae181f035af5d12ba9d_SeqId_44_ into 2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:54:59,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vtiosgts5ed5rgg788miagjee527jjsp0u078teg0d0grhdfmbb8p5nfgsthcjcb/family1/ad40ac4a924c410b8939155c223b560f into 2f876068b9705192eb71b2d2aa908931/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/8eb1fdb4bcb14ae181f035af5d12ba9d_SeqId_44_) 2023-02-08 23:55:00,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__vtiosgts5ed5rgg788miagjee527jjsp0u078teg0d0grhdfmbb8p5nfgsthcjcb/family1/ad40ac4a924c410b8939155c223b560f 2023-02-08 23:55:00,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 23:55:00,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 23:55:00,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 2f876068b9705192eb71b2d2aa908931/family1 is initiating minor compaction 2023-02-08 23:55:00,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4e8ef71; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:00,003 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2f876068b9705192eb71b2d2aa908931/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:00,003 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/80c509f63cbd4486995736daffd6776e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/941187bb1e034d9e9124547d1817f18d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a04a4d7c03604b4da24fc17aa23ecfb6, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/983e19fcd204433882ab0a7da4bad95f, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/4388d1d37d834ea1b1c713fb49d8736d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/02a163a6b79e4dd5bf2d11bdc03e0d32, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/ecc77f4cce794a5490073c659f8497cf, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/d3b754f744cc4ef293a364e5572ceab0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a4eea1b433144d328e6d4cd32820bee6, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/8eb1fdb4bcb14ae181f035af5d12ba9d_SeqId_44_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp, totalSize=50.3 K 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 80c509f63cbd4486995736daffd6776e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 941187bb1e034d9e9124547d1817f18d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting a04a4d7c03604b4da24fc17aa23ecfb6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 983e19fcd204433882ab0a7da4bad95f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 4388d1d37d834ea1b1c713fb49d8736d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 02a163a6b79e4dd5bf2d11bdc03e0d32, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting ecc77f4cce794a5490073c659f8497cf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting d3b754f744cc4ef293a364e5572ceab0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting a4eea1b433144d328e6d4cd32820bee6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 23:55:00,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 8eb1fdb4bcb14ae181f035af5d12ba9d_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 23:55:00,004 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:00,005 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:00,006 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228f91c85e5aabe495faf1612671e4c228b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228f91c85e5aabe495faf1612671e4c228b 2023-02-08 23:55:00,007 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231ea0037d9f722443abef635942ffb8b34 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231ea0037d9f722443abef635942ffb8b34 2023-02-08 23:55:00,009 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010110dc2a6acb4b4058a9a8d6af8cf85b02 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010110dc2a6acb4b4058a9a8d6af8cf85b02 2023-02-08 23:55:00,010 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103b831623478b94abbbaeb5ace574e3b95 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103b831623478b94abbbaeb5ace574e3b95 2023-02-08 23:55:00,010 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-08 23:55:00,010 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 23:55:00,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:00,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:00,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:00,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:00,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:00,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:00,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:00,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:00,044 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2f876068b9705192eb71b2d2aa908931#family1#compaction#72 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:00,864 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/03ce5aba466c4ba49b5633b9ba88f522 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/03ce5aba466c4ba49b5633b9ba88f522 2023-02-08 23:55:00,868 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2f876068b9705192eb71b2d2aa908931/family1 of 2f876068b9705192eb71b2d2aa908931 into 03ce5aba466c4ba49b5633b9ba88f522(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:00,868 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:55:00,868 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675900500002; duration=0sec 2023-02-08 23:55:00,868 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:00,872 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205935c1129b7aa4301913d4c42d08bb4b7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205935c1129b7aa4301913d4c42d08bb4b7 2023-02-08 23:55:00,872 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:00,888 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:00,888 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:00,895 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b3107394c55645a0abb946eebea38b75 first=Optional[r00] last=Optional[r2] 2023-02-08 23:55:00,895 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b3107394c55645a0abb946eebea38b75}] 2023-02-08 23:55:00,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b3107394c55645a0abb946eebea38b75 for inclusion in 2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:55:00,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-08 23:55:00,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:55:00,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:55:00,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b3107394c55645a0abb946eebea38b75 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ol9mndrka6bge4f4l3rs772gn3052n4l0dcba05kbt98sc6go8s585tvn6mrram9/family1/b3107394c55645a0abb946eebea38b75 2023-02-08 23:55:00,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ol9mndrka6bge4f4l3rs772gn3052n4l0dcba05kbt98sc6go8s585tvn6mrram9/family1/b3107394c55645a0abb946eebea38b75 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/cb0db2111b3f4d7e952aab0a9ff8ace1_SeqId_47_ 2023-02-08 23:55:00,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ol9mndrka6bge4f4l3rs772gn3052n4l0dcba05kbt98sc6go8s585tvn6mrram9/family1/b3107394c55645a0abb946eebea38b75 into 2f876068b9705192eb71b2d2aa908931/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/cb0db2111b3f4d7e952aab0a9ff8ace1_SeqId_47_ - updating store file list. 2023-02-08 23:55:00,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/cb0db2111b3f4d7e952aab0a9ff8ace1_SeqId_47_ into 2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:55:00,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ol9mndrka6bge4f4l3rs772gn3052n4l0dcba05kbt98sc6go8s585tvn6mrram9/family1/b3107394c55645a0abb946eebea38b75 into 2f876068b9705192eb71b2d2aa908931/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/cb0db2111b3f4d7e952aab0a9ff8ace1_SeqId_47_) 2023-02-08 23:55:00,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ol9mndrka6bge4f4l3rs772gn3052n4l0dcba05kbt98sc6go8s585tvn6mrram9/family1/b3107394c55645a0abb946eebea38b75 2023-02-08 23:55:00,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 23:55:00,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 23:55:00,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 2f876068b9705192eb71b2d2aa908931/family1 is initiating minor compaction (all files) 2023-02-08 23:55:00,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@63415204; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:00,907 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2f876068b9705192eb71b2d2aa908931/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:00,908 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/1f45f0439135472c873a4566842b953c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/03ce5aba466c4ba49b5633b9ba88f522, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/cb0db2111b3f4d7e952aab0a9ff8ace1_SeqId_47_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp, totalSize=388.1 K 2023-02-08 23:55:00,908 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 1f45f0439135472c873a4566842b953c, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838754 2023-02-08 23:55:00,908 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 03ce5aba466c4ba49b5633b9ba88f522, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238754 2023-02-08 23:55:00,909 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting cb0db2111b3f4d7e952aab0a9ff8ace1_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878838754 2023-02-08 23:55:00,909 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:00,910 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:00,911 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130a750985bfb52426e81b886deac2f02cc to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130a750985bfb52426e81b886deac2f02cc 2023-02-08 23:55:00,912 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c13f0b282c6648b8af265f35f226204e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201c13f0b282c6648b8af265f35f226204e 2023-02-08 23:55:00,913 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120573f9b2be7d6e4798b0a79a04790d6843 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120573f9b2be7d6e4798b0a79a04790d6843 2023-02-08 23:55:00,913 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-08 23:55:00,913 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 23:55:00,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:00,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:00,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:55:00,915 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 23:55:00,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8130 sec 2023-02-08 23:55:00,923 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2f876068b9705192eb71b2d2aa908931#family1#compaction#73 average throughput is 25.66 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:00,965 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/80c509f63cbd4486995736daffd6776e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/941187bb1e034d9e9124547d1817f18d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a04a4d7c03604b4da24fc17aa23ecfb6, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/983e19fcd204433882ab0a7da4bad95f, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/4388d1d37d834ea1b1c713fb49d8736d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/02a163a6b79e4dd5bf2d11bdc03e0d32, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/ecc77f4cce794a5490073c659f8497cf, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/d3b754f744cc4ef293a364e5572ceab0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a4eea1b433144d328e6d4cd32820bee6, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/8eb1fdb4bcb14ae181f035af5d12ba9d_SeqId_44_] to archive 2023-02-08 23:55:00,966 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:00,968 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/80c509f63cbd4486995736daffd6776e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/80c509f63cbd4486995736daffd6776e 2023-02-08 23:55:00,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/941187bb1e034d9e9124547d1817f18d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/941187bb1e034d9e9124547d1817f18d 2023-02-08 23:55:00,971 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a04a4d7c03604b4da24fc17aa23ecfb6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a04a4d7c03604b4da24fc17aa23ecfb6 2023-02-08 23:55:00,972 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/983e19fcd204433882ab0a7da4bad95f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/983e19fcd204433882ab0a7da4bad95f 2023-02-08 23:55:00,974 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/4388d1d37d834ea1b1c713fb49d8736d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/4388d1d37d834ea1b1c713fb49d8736d 2023-02-08 23:55:00,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/02a163a6b79e4dd5bf2d11bdc03e0d32 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/02a163a6b79e4dd5bf2d11bdc03e0d32 2023-02-08 23:55:00,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/ecc77f4cce794a5490073c659f8497cf to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/ecc77f4cce794a5490073c659f8497cf 2023-02-08 23:55:00,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/d3b754f744cc4ef293a364e5572ceab0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/d3b754f744cc4ef293a364e5572ceab0 2023-02-08 23:55:00,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a4eea1b433144d328e6d4cd32820bee6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/a4eea1b433144d328e6d4cd32820bee6 2023-02-08 23:55:00,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/8eb1fdb4bcb14ae181f035af5d12ba9d_SeqId_44_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/8eb1fdb4bcb14ae181f035af5d12ba9d_SeqId_44_ 2023-02-08 23:55:00,987 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:00,987 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:00,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 23:55:00,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:55:01,398 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-08 23:55:01,399 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:01,399 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-08 23:55:01,401 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-08 23:55:01,404 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, REOPEN/MOVE}] 2023-02-08 23:55:01,405 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, REOPEN/MOVE 2023-02-08 23:55:01,405 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:01,406 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900501405"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900501405"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900501405"}]},"ts":"1675900501405"} 2023-02-08 23:55:01,407 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:01,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:01,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2f876068b9705192eb71b2d2aa908931, disabling compactions & flushes 2023-02-08 23:55:01,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:02,146 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/.tmp/family1/49ebe1024aea436dab8c523759a14e78 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/49ebe1024aea436dab8c523759a14e78 2023-02-08 23:55:02,150 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 2f876068b9705192eb71b2d2aa908931/family1 of 2f876068b9705192eb71b2d2aa908931 into 49ebe1024aea436dab8c523759a14e78(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:02,150 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:55:02,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:02,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:02,150 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1675900500907; duration=1sec 2023-02-08 23:55:02,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. after waiting 0 ms 2023-02-08 23:55:02,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:02,150 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:02,153 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/1f45f0439135472c873a4566842b953c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/03ce5aba466c4ba49b5633b9ba88f522, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/cb0db2111b3f4d7e952aab0a9ff8ace1_SeqId_47_] to archive 2023-02-08 23:55:02,153 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:02,155 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/1f45f0439135472c873a4566842b953c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/1f45f0439135472c873a4566842b953c 2023-02-08 23:55:02,156 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/03ce5aba466c4ba49b5633b9ba88f522 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/03ce5aba466c4ba49b5633b9ba88f522 2023-02-08 23:55:02,157 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/cb0db2111b3f4d7e952aab0a9ff8ace1_SeqId_47_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/cb0db2111b3f4d7e952aab0a9ff8ace1_SeqId_47_ 2023-02-08 23:55:02,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-08 23:55:02,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:02,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2f876068b9705192eb71b2d2aa908931: Waiting for close lock at 1675900501558Running coprocessor pre-close hooks at 1675900501558Disabling compacts and flushes for region at 1675900501558Disabling writes for close at 1675900502150 (+592 ms)Writing region close event to WAL at 1675900502158 (+8 ms)Running coprocessor post-close hooks at 1675900502161 (+3 ms)Closed at 1675900502161 2023-02-08 23:55:02,161 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 2f876068b9705192eb71b2d2aa908931 to self. 2023-02-08 23:55:02,163 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:02,163 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=CLOSED 2023-02-08 23:55:02,163 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900502163"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900502163"}]},"ts":"1675900502163"} 2023-02-08 23:55:02,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-08 23:55:02,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768 in 757 msec 2023-02-08 23:55:02,166 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=true 2023-02-08 23:55:02,316 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:02,316 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900502316"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900502316"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900502316"}]},"ts":"1675900502316"} 2023-02-08 23:55:02,318 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:02,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:02,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2f876068b9705192eb71b2d2aa908931, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:55:02,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:02,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:02,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:02,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:02,474 INFO [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:02,475 DEBUG [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:55:02,475 DEBUG [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:55:02,475 INFO [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f876068b9705192eb71b2d2aa908931 columnFamilyName family1 2023-02-08 23:55:02,479 DEBUG [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/49ebe1024aea436dab8c523759a14e78 2023-02-08 23:55:02,479 INFO [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] regionserver.HStore(310): Store=2f876068b9705192eb71b2d2aa908931/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:02,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:02,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:02,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:02,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2f876068b9705192eb71b2d2aa908931; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70672718, jitterRate=0.053105562925338745}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:55:02,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2f876068b9705192eb71b2d2aa908931: Running coprocessor pre-open hook at 1675900502473Writing region info on filesystem at 1675900502473Initializing all the Stores at 1675900502474 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675900502474Cleaning up temporary data from old regions at 1675900502481 (+7 ms)Cleaning up detritus from prior splits at 1675900502482 (+1 ms)Running coprocessor post-open hooks at 1675900502484 (+2 ms)Region opened successfully at 1675900502484 2023-02-08 23:55:02,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931., pid=63, masterSystemTime=1675900502469 2023-02-08 23:55:02,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:02,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:02,486 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:02,486 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900502486"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900502486"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900502486"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900502486"}]},"ts":"1675900502486"} 2023-02-08 23:55:02,489 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-08 23:55:02,489 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768 in 169 msec 2023-02-08 23:55:02,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-08 23:55:02,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, REOPEN/MOVE in 1.0850 sec 2023-02-08 23:55:02,492 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-08 23:55:02,492 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0890 sec 2023-02-08 23:55:02,493 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5030 sec 2023-02-08 23:55:02,494 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:02,494 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:02,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-08 23:55:02,494 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:02,494 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:02,495 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:02,495 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:02,495 INFO [Listener at localhost/41521] compactions.TestMobCompactor(278): alter status finished 2023-02-08 23:55:02,496 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:55:02,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 23:55:02,498 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 23:55:02,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:55:02,498 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901102498 2023-02-08 23:55:02,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:55:02,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-08 23:55:02,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:02,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x4278bf06 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:55:02,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2a4d6820, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:55:02,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:02,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:02,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:02,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:02,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:03,326 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128712937a627db4ab9be45e00480897417 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128712937a627db4ab9be45e00480897417 2023-02-08 23:55:03,327 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:03,339 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:55:03,340 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56360, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:55:03,346 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:03,346 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:03,347 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:55:03,348 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48306, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:55:03,355 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a90753c6df174abc8cb526687581349b first=Optional[r05] last=Optional[r8] 2023-02-08 23:55:03,356 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=51 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a90753c6df174abc8cb526687581349b}] 2023-02-08 23:55:03,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a90753c6df174abc8cb526687581349b for inclusion in 2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:55:03,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-08 23:55:03,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:55:03,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 2f876068b9705192eb71b2d2aa908931: 2023-02-08 23:55:03,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a90753c6df174abc8cb526687581349b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__otgla3u4ofdn60ptkl6f2nn8njnd2jamvr53o7gpkafkuhebkcdafm82ihmp6q3v/family1/a90753c6df174abc8cb526687581349b 2023-02-08 23:55:03,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__otgla3u4ofdn60ptkl6f2nn8njnd2jamvr53o7gpkafkuhebkcdafm82ihmp6q3v/family1/a90753c6df174abc8cb526687581349b as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/f7ba49224df34810a3875853f7895dee_SeqId_53_ 2023-02-08 23:55:03,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__otgla3u4ofdn60ptkl6f2nn8njnd2jamvr53o7gpkafkuhebkcdafm82ihmp6q3v/family1/a90753c6df174abc8cb526687581349b into 2f876068b9705192eb71b2d2aa908931/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/f7ba49224df34810a3875853f7895dee_SeqId_53_ - updating store file list. 2023-02-08 23:55:03,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/f7ba49224df34810a3875853f7895dee_SeqId_53_ into 2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:55:03,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__otgla3u4ofdn60ptkl6f2nn8njnd2jamvr53o7gpkafkuhebkcdafm82ihmp6q3v/family1/a90753c6df174abc8cb526687581349b into 2f876068b9705192eb71b2d2aa908931/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/f7ba49224df34810a3875853f7895dee_SeqId_53_) 2023-02-08 23:55:03,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__otgla3u4ofdn60ptkl6f2nn8njnd2jamvr53o7gpkafkuhebkcdafm82ihmp6q3v/family1/a90753c6df174abc8cb526687581349b 2023-02-08 23:55:03,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:03,371 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:03,372 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120ee3ee527c57a41f1ad72927a3df7d08f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120ee3ee527c57a41f1ad72927a3df7d08f 2023-02-08 23:55:03,374 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128a47d97376ec64afea3863d5b66b8deac to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128a47d97376ec64afea3863d5b66b8deac 2023-02-08 23:55:03,374 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-08 23:55:03,374 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 23:55:03,374 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:03,374 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:03,374 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:03,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:03,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:55:03,375 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 23:55:03,377 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 879 msec 2023-02-08 23:55:03,430 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:03,430 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:03,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-08 23:55:03,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:55:03,840 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-08 23:55:03,841 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-08 23:55:03,841 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-08 23:55:03,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-08 23:55:03,845 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, REOPEN/MOVE}] 2023-02-08 23:55:03,845 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, REOPEN/MOVE 2023-02-08 23:55:03,846 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:03,846 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900503846"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900503846"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900503846"}]},"ts":"1675900503846"} 2023-02-08 23:55:03,847 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:03,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:03,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2f876068b9705192eb71b2d2aa908931, disabling compactions & flushes 2023-02-08 23:55:03,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:03,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:03,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. after waiting 0 ms 2023-02-08 23:55:03,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:04,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-08 23:55:04,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:04,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2f876068b9705192eb71b2d2aa908931: Waiting for close lock at 1675900503999Running coprocessor pre-close hooks at 1675900503999Disabling compacts and flushes for region at 1675900503999Disabling writes for close at 1675900503999Writing region close event to WAL at 1675900504001 (+2 ms)Running coprocessor post-close hooks at 1675900504004 (+3 ms)Closed at 1675900504004 2023-02-08 23:55:04,005 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 2f876068b9705192eb71b2d2aa908931 to self. 2023-02-08 23:55:04,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:04,006 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=CLOSED 2023-02-08 23:55:04,006 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900504006"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900504006"}]},"ts":"1675900504006"} 2023-02-08 23:55:04,008 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-08 23:55:04,008 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768 in 160 msec 2023-02-08 23:55:04,009 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=true 2023-02-08 23:55:04,159 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:04,159 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900504159"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900504159"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900504159"}]},"ts":"1675900504159"} 2023-02-08 23:55:04,161 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:04,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:04,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2f876068b9705192eb71b2d2aa908931, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:55:04,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:04,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:04,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:04,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:04,316 INFO [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:04,317 DEBUG [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:55:04,317 DEBUG [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1 2023-02-08 23:55:04,318 INFO [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f876068b9705192eb71b2d2aa908931 columnFamilyName family1 2023-02-08 23:55:04,322 DEBUG [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/49ebe1024aea436dab8c523759a14e78 2023-02-08 23:55:04,325 DEBUG [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/family1/f7ba49224df34810a3875853f7895dee_SeqId_53_ 2023-02-08 23:55:04,325 INFO [StoreOpener-2f876068b9705192eb71b2d2aa908931-1] regionserver.HStore(310): Store=2f876068b9705192eb71b2d2aa908931/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:04,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:04,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:04,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:55:04,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2f876068b9705192eb71b2d2aa908931; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66834001, jitterRate=-0.004095777869224548}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:55:04,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2f876068b9705192eb71b2d2aa908931: Running coprocessor pre-open hook at 1675900504315Writing region info on filesystem at 1675900504315Initializing all the Stores at 1675900504316 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1675900504316Cleaning up temporary data from old regions at 1675900504327 (+11 ms)Cleaning up detritus from prior splits at 1675900504328 (+1 ms)Running coprocessor post-open hooks at 1675900504330 (+2 ms)Region opened successfully at 1675900504330 2023-02-08 23:55:04,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931., pid=69, masterSystemTime=1675900504312 2023-02-08 23:55:04,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:04,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:55:04,333 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2f876068b9705192eb71b2d2aa908931, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:04,333 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675900504333"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900504333"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900504333"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900504333"}]},"ts":"1675900504333"} 2023-02-08 23:55:04,335 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-08 23:55:04,336 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 2f876068b9705192eb71b2d2aa908931, server=jenkins-hbase4.apache.org,42703,1675900414768 in 173 msec 2023-02-08 23:55:04,337 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-08 23:55:04,337 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2f876068b9705192eb71b2d2aa908931, REOPEN/MOVE in 491 msec 2023-02-08 23:55:04,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-08 23:55:04,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 495 msec 2023-02-08 23:55:04,340 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 908 msec 2023-02-08 23:55:04,341 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:04,341 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:04,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-08 23:55:04,342 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:04,342 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:04,342 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:04,342 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:04,343 INFO [Listener at localhost/41521] compactions.TestMobCompactor(278): alter status finished 2023-02-08 23:55:04,344 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 23:55:04,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 23:55:04,346 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 23:55:04,346 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:55:04,346 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901104346 2023-02-08 23:55:04,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:55:04,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-08 23:55:04,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:04,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x67693ffb to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:55:04,352 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5fda74d2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:55:04,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:04,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:04,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:04,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:04,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:04,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:04,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:04,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 23:55:04,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 23:55:04,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:55:04,353 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 23:55:04,354 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 9 msec 2023-02-08 23:55:04,399 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=242 (was 249), OpenFileDescriptor=619 (was 633), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=142 (was 183), ProcessCount=168 (was 168), AvailableMemoryMB=3229 (was 3293) 2023-02-08 23:55:04,410 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=242, OpenFileDescriptor=619, MaxFileDescriptor=60000, SystemLoadAverage=142, ProcessCount=168, AvailableMemoryMB=3229 2023-02-08 23:55:04,411 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:04,411 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:04,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 23:55:04,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-08 23:55:04,414 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:55:04,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-08 23:55:04,414 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:04,414 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:04,414 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:55:04,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 23:55:04,416 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:04,416 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836 empty. 2023-02-08 23:55:04,416 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:04,416 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-08 23:55:04,515 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:04,515 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:04,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 23:55:04,716 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:04,716 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:04,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 23:55:04,825 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:04,826 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 86f906b69ef7ff6ae8f6e322ddced836, NAME => 'testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:55:05,017 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:05,018 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:05,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 23:55:05,231 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:05,232 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing 86f906b69ef7ff6ae8f6e322ddced836, disabling compactions & flushes 2023-02-08 23:55:05,232 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:05,232 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:05,232 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. after waiting 0 ms 2023-02-08 23:55:05,232 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:05,232 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:05,232 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 86f906b69ef7ff6ae8f6e322ddced836: Waiting for close lock at 1675900505232Disabling compacts and flushes for region at 1675900505232Disabling writes for close at 1675900505232Writing region close event to WAL at 1675900505232Closed at 1675900505232 2023-02-08 23:55:05,234 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:55:05,234 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900505234"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900505234"}]},"ts":"1675900505234"} 2023-02-08 23:55:05,236 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:55:05,236 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:55:05,236 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900505236"}]},"ts":"1675900505236"} 2023-02-08 23:55:05,237 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-08 23:55:05,240 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=86f906b69ef7ff6ae8f6e322ddced836, ASSIGN}] 2023-02-08 23:55:05,241 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=86f906b69ef7ff6ae8f6e322ddced836, ASSIGN 2023-02-08 23:55:05,242 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=86f906b69ef7ff6ae8f6e322ddced836, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:55:05,392 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=86f906b69ef7ff6ae8f6e322ddced836, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:05,393 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900505392"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900505392"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900505392"}]},"ts":"1675900505392"} 2023-02-08 23:55:05,394 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 86f906b69ef7ff6ae8f6e322ddced836, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:05,518 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:05,519 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:05,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 23:55:05,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:05,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 86f906b69ef7ff6ae8f6e322ddced836, NAME => 'testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:55:05,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-08 23:55:05,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-08 23:55:05,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:05,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:05,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:05,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:05,551 INFO [StoreOpener-86f906b69ef7ff6ae8f6e322ddced836-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:05,552 DEBUG [StoreOpener-86f906b69ef7ff6ae8f6e322ddced836-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1 2023-02-08 23:55:05,553 DEBUG [StoreOpener-86f906b69ef7ff6ae8f6e322ddced836-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1 2023-02-08 23:55:05,553 INFO [StoreOpener-86f906b69ef7ff6ae8f6e322ddced836-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86f906b69ef7ff6ae8f6e322ddced836 columnFamilyName f1 2023-02-08 23:55:05,554 INFO [StoreOpener-86f906b69ef7ff6ae8f6e322ddced836-1] regionserver.HStore(310): Store=86f906b69ef7ff6ae8f6e322ddced836/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:05,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:05,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:05,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:05,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:05,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 86f906b69ef7ff6ae8f6e322ddced836; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74492357, jitterRate=0.11002261936664581}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:55:05,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 86f906b69ef7ff6ae8f6e322ddced836: Running coprocessor pre-open hook at 1675900505550Writing region info on filesystem at 1675900505550Initializing all the Stores at 1675900505551 (+1 ms)Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900505551Cleaning up temporary data from old regions at 1675900505555 (+4 ms)Cleaning up detritus from prior splits at 1675900505556 (+1 ms)Running coprocessor post-open hooks at 1675900505559 (+3 ms)Region opened successfully at 1675900505559 2023-02-08 23:55:05,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836., pid=73, masterSystemTime=1675900505546 2023-02-08 23:55:05,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:05,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:05,561 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=86f906b69ef7ff6ae8f6e322ddced836, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:05,561 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900505561"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900505561"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900505561"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900505561"}]},"ts":"1675900505561"} 2023-02-08 23:55:05,563 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-08 23:55:05,563 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 86f906b69ef7ff6ae8f6e322ddced836, server=jenkins-hbase4.apache.org,42703,1675900414768 in 168 msec 2023-02-08 23:55:05,565 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-08 23:55:05,565 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=86f906b69ef7ff6ae8f6e322ddced836, ASSIGN in 323 msec 2023-02-08 23:55:05,565 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:55:05,565 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900505565"}]},"ts":"1675900505565"} 2023-02-08 23:55:05,566 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-08 23:55:05,569 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:55:05,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1570 sec 2023-02-08 23:55:05,929 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-08 23:55:06,520 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:06,520 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:06,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 23:55:06,520 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-08 23:55:06,521 DEBUG [Listener at localhost/41521] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-08 23:55:06,522 INFO [Listener at localhost/41521] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-08 23:55:06,525 INFO [Listener at localhost/41521] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-08 23:55:06,525 INFO [Listener at localhost/41521] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-08 23:55:06,525 INFO [Listener at localhost/41521] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-08 23:55:06,529 INFO [Listener at localhost/41521] regionserver.HRegion(2744): Flushing 86f906b69ef7ff6ae8f6e322ddced836 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-08 23:55:06,940 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020886db41a484304920a1e64db090309b1d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023020886db41a484304920a1e64db090309b1d 2023-02-08 23:55:07,346 INFO [Listener at localhost/41521] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/.tmp/f1/89562efa622446daa250b4ce36cfa5da 2023-02-08 23:55:07,351 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/.tmp/f1/89562efa622446daa250b4ce36cfa5da as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da 2023-02-08 23:55:07,354 INFO [Listener at localhost/41521] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da, entries=1, sequenceid=5, filesize=4.9 K 2023-02-08 23:55:07,355 INFO [Listener at localhost/41521] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 86f906b69ef7ff6ae8f6e322ddced836 in 826ms, sequenceid=5, compaction requested=false 2023-02-08 23:55:07,356 DEBUG [Listener at localhost/41521] regionserver.HRegion(2445): Flush status journal for 86f906b69ef7ff6ae8f6e322ddced836: 2023-02-08 23:55:07,358 INFO [Listener at localhost/41521] regionserver.HRegion(2744): Flushing 86f906b69ef7ff6ae8f6e322ddced836 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-08 23:55:07,702 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:55:07,769 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302087bba4988af8e4eca964258daa1943dc6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302087bba4988af8e4eca964258daa1943dc6 2023-02-08 23:55:08,174 INFO [Listener at localhost/41521] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/.tmp/f1/950d16e6a739464ba7c44b8e22791a86 2023-02-08 23:55:08,178 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/.tmp/f1/950d16e6a739464ba7c44b8e22791a86 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86 2023-02-08 23:55:08,181 INFO [Listener at localhost/41521] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86, entries=1, sequenceid=9, filesize=4.9 K 2023-02-08 23:55:08,182 INFO [Listener at localhost/41521] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 86f906b69ef7ff6ae8f6e322ddced836 in 824ms, sequenceid=9, compaction requested=false 2023-02-08 23:55:08,182 DEBUG [Listener at localhost/41521] regionserver.HRegion(2445): Flush status journal for 86f906b69ef7ff6ae8f6e322ddced836: 2023-02-08 23:55:08,183 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-08 23:55:08,184 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-08 23:55:08,185 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-08 23:55:08,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:55:08,186 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901108186 2023-02-08 23:55:08,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 23:55:08,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-08 23:55:08,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-08 23:55:08,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x5b06711e to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:55:08,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@57f6e248, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:55:08,191 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='testGetAfterCompaction' and column='f1' 2023-02-08 23:55:08,235 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,235 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,285 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,285 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,336 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,336 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,387 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,387 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,438 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,438 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,489 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,489 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,540 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,540 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,591 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,591 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,642 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,642 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,693 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,693 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,743 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,744 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,794 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,794 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,845 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,845 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,896 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,896 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,947 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,947 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,998 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:08,998 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,014 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020846223a01d3014833bd371797a3c8fea8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023020846223a01d3014833bd371797a3c8fea8 2023-02-08 23:55:09,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testGetAfterCompaction' column='f1' 2023-02-08 23:55:09,026 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:55:09,028 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58366, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:55:09,033 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:09,033 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:09,033 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:55:09,035 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33004, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:55:09,043 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/f1/8961350f24a44251bae2352dbfdbccba first=Optional[r1] last=Optional[r2] 2023-02-08 23:55:09,043 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{f1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/f1/8961350f24a44251bae2352dbfdbccba}] 2023-02-08 23:55:09,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/f1/8961350f24a44251bae2352dbfdbccba for inclusion in 86f906b69ef7ff6ae8f6e322ddced836/f1 2023-02-08 23:55:09,048 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,048 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-08 23:55:09,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:55:09,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 86f906b69ef7ff6ae8f6e322ddced836: 2023-02-08 23:55:09,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/f1/8961350f24a44251bae2352dbfdbccba to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testGetAfterCompaction__19gr5boqn122tbi4cq62rmr460hdkpk53jjt7606b3pqp2vi56t6eb712duc6nlk/f1/8961350f24a44251bae2352dbfdbccba 2023-02-08 23:55:09,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testGetAfterCompaction__19gr5boqn122tbi4cq62rmr460hdkpk53jjt7606b3pqp2vi56t6eb712duc6nlk/f1/8961350f24a44251bae2352dbfdbccba as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_ 2023-02-08 23:55:09,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testGetAfterCompaction__19gr5boqn122tbi4cq62rmr460hdkpk53jjt7606b3pqp2vi56t6eb712duc6nlk/f1/8961350f24a44251bae2352dbfdbccba into 86f906b69ef7ff6ae8f6e322ddced836/f1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_ - updating store file list. 2023-02-08 23:55:09,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_ into 86f906b69ef7ff6ae8f6e322ddced836/f1 2023-02-08 23:55:09,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testGetAfterCompaction__19gr5boqn122tbi4cq62rmr460hdkpk53jjt7606b3pqp2vi56t6eb712duc6nlk/f1/8961350f24a44251bae2352dbfdbccba into 86f906b69ef7ff6ae8f6e322ddced836/f1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_) 2023-02-08 23:55:09,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testGetAfterCompaction__19gr5boqn122tbi4cq62rmr460hdkpk53jjt7606b3pqp2vi56t6eb712duc6nlk/f1/8961350f24a44251bae2352dbfdbccba 2023-02-08 23:55:09,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 86f906b69ef7ff6ae8f6e322ddced836/f1 is initiating minor compaction 2023-02-08 23:55:09,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6e83bab1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:09,056 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 86f906b69ef7ff6ae8f6e322ddced836/f1 in testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:09,056 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/.tmp, totalSize=10.0 K 2023-02-08 23:55:09,056 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 950d16e6a739464ba7c44b8e22791a86, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 23:55:09,056 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 4df5109e1a4d46e5b425c454d52d7925_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 23:55:09,058 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testGetAfterCompaction' column='f1' 2023-02-08 23:55:09,058 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:09,060 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302087bba4988af8e4eca964258daa1943dc6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302087bba4988af8e4eca964258daa1943dc6 2023-02-08 23:55:09,061 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023020886db41a484304920a1e64db090309b1d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023020886db41a484304920a1e64db090309b1d 2023-02-08 23:55:09,061 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-08 23:55:09,061 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 23:55:09,062 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='testGetAfterCompaction' and column='f1' 2023-02-08 23:55:09,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-08 23:55:09,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:55:09,063 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-08 23:55:09,064 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 879 msec 2023-02-08 23:55:09,067 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 86f906b69ef7ff6ae8f6e322ddced836#f1#compaction#76 average throughput is 0.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:09,099 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,099 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,099 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,103 DEBUG [Listener at localhost/41521] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 23:55:09,106 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58380, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 23:55:09,106 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42703] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:09,107 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-08 23:55:09,107 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-08 23:55:09,107 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42703] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-08 23:55:09,107 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42703] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. because compaction request was cancelled 2023-02-08 23:55:09,158 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,158 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,158 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,209 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,209 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,209 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,260 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,260 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,260 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,310 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,311 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,311 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,361 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,361 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,361 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,412 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,412 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,412 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,463 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,463 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,463 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,514 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,514 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,514 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,565 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,565 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,565 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,616 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,616 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,616 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,666 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,667 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,667 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,717 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,717 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,717 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,768 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,768 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,768 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,819 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,819 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,819 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,870 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,870 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,870 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,879 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/.tmp/f1/1cb6b21a43f14aa0b277e5cf7a1ef612 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/1cb6b21a43f14aa0b277e5cf7a1ef612 2023-02-08 23:55:09,884 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 86f906b69ef7ff6ae8f6e322ddced836/f1 of 86f906b69ef7ff6ae8f6e322ddced836 into 1cb6b21a43f14aa0b277e5cf7a1ef612(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:09,884 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 86f906b69ef7ff6ae8f6e322ddced836: 2023-02-08 23:55:09,884 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1675900509056; duration=0sec 2023-02-08 23:55:09,884 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:09,921 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/1cb6b21a43f14aa0b277e5cf7a1ef612; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675900509875; access_time=1675900509061; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,921 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675900509010; access_time=1675900508202; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 23:55:09,921 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,921 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900508174; access_time=1675900507359; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,961 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_] to archive 2023-02-08 23:55:09,962 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:09,963 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/950d16e6a739464ba7c44b8e22791a86 2023-02-08 23:55:09,964 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/4df5109e1a4d46e5b425c454d52d7925_SeqId_12_ 2023-02-08 23:55:09,972 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/1cb6b21a43f14aa0b277e5cf7a1ef612; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675900509875; access_time=1675900509061; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:09,972 INFO [Listener at localhost/41521] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675900507345; access_time=1675900506531; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 23:55:10,087 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,087 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,087 INFO [Listener at localhost/41521] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-08 23:55:10,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-02-08 23:55:10,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-08 23:55:10,100 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900510100"}]},"ts":"1675900510100"} 2023-02-08 23:55:10,100 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,101 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,101 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-08 23:55:10,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-08 23:55:10,103 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-08 23:55:10,104 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=86f906b69ef7ff6ae8f6e322ddced836, UNASSIGN}] 2023-02-08 23:55:10,105 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=86f906b69ef7ff6ae8f6e322ddced836, UNASSIGN 2023-02-08 23:55:10,106 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=86f906b69ef7ff6ae8f6e322ddced836, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:10,106 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900510106"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900510106"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900510106"}]},"ts":"1675900510106"} 2023-02-08 23:55:10,107 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 86f906b69ef7ff6ae8f6e322ddced836, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:10,201 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,202 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-08 23:55:10,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:10,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 86f906b69ef7ff6ae8f6e322ddced836, disabling compactions & flushes 2023-02-08 23:55:10,259 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:10,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:10,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. after waiting 0 ms 2023-02-08 23:55:10,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:10,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-08 23:55:10,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-08 23:55:10,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836. 2023-02-08 23:55:10,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 86f906b69ef7ff6ae8f6e322ddced836: Waiting for close lock at 1675900510259Running coprocessor pre-close hooks at 1675900510259Disabling compacts and flushes for region at 1675900510259Disabling writes for close at 1675900510259Writing region close event to WAL at 1675900510261 (+2 ms)Running coprocessor post-close hooks at 1675900510265 (+4 ms)Closed at 1675900510265 2023-02-08 23:55:10,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:10,267 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=86f906b69ef7ff6ae8f6e322ddced836, regionState=CLOSED 2023-02-08 23:55:10,267 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675900510267"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900510267"}]},"ts":"1675900510267"} 2023-02-08 23:55:10,270 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-08 23:55:10,270 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 86f906b69ef7ff6ae8f6e322ddced836, server=jenkins-hbase4.apache.org,42703,1675900414768 in 161 msec 2023-02-08 23:55:10,271 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-08 23:55:10,271 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=86f906b69ef7ff6ae8f6e322ddced836, UNASSIGN in 166 msec 2023-02-08 23:55:10,271 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900510271"}]},"ts":"1675900510271"} 2023-02-08 23:55:10,272 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-08 23:55:10,278 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-08 23:55:10,279 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 185 msec 2023-02-08 23:55:10,402 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,403 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-08 23:55:10,404 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-08 23:55:10,404 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,404 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-02-08 23:55:10,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 23:55:10,415 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 23:55:10,416 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 23:55:10,417 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,417 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-08 23:55:10,419 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:10,421 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1, FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/recovered.edits] 2023-02-08 23:55:10,424 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/1cb6b21a43f14aa0b277e5cf7a1ef612 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/1cb6b21a43f14aa0b277e5cf7a1ef612 2023-02-08 23:55:10,425 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/f1/89562efa622446daa250b4ce36cfa5da 2023-02-08 23:55:10,427 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/recovered.edits/15.seqid to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836/recovered.edits/15.seqid 2023-02-08 23:55:10,428 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testGetAfterCompaction/86f906b69ef7ff6ae8f6e322ddced836 2023-02-08 23:55:10,428 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-08 23:55:10,428 DEBUG [PEWorker-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-08 23:55:10,429 DEBUG [PEWorker-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-08 23:55:10,432 DEBUG [PEWorker-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023020846223a01d3014833bd371797a3c8fea8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023020846223a01d3014833bd371797a3c8fea8 2023-02-08 23:55:10,432 DEBUG [PEWorker-5] backup.HFileArchiver(596): Deleted hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-08 23:55:10,434 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 23:55:10,455 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-08 23:55:10,457 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-08 23:55:10,458 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 23:55:10,458 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-08 23:55:10,458 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900510458"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:10,459 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-08 23:55:10,459 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 86f906b69ef7ff6ae8f6e322ddced836, NAME => 'testGetAfterCompaction,,1675900504412.86f906b69ef7ff6ae8f6e322ddced836.', STARTKEY => '', ENDKEY => ''}] 2023-02-08 23:55:10,460 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-08 23:55:10,460 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675900510460"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:10,461 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-08 23:55:10,462 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 23:55:10,463 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 53 msec 2023-02-08 23:55:10,517 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,518 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-08 23:55:10,519 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-08 23:55:10,531 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=245 (was 242) - Thread LEAK? -, OpenFileDescriptor=619 (was 619), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=131 (was 142), ProcessCount=168 (was 168), AvailableMemoryMB=3226 (was 3229) 2023-02-08 23:55:10,541 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=245, OpenFileDescriptor=619, MaxFileDescriptor=60000, SystemLoadAverage=131, ProcessCount=168, AvailableMemoryMB=3225 2023-02-08 23:55:10,543 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,543 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 23:55:10,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-08 23:55:10,546 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:55:10,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-08 23:55:10,546 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,546 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:55:10,546 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 23:55:10,549 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:10,549 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:10,549 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:10,549 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5 empty. 2023-02-08 23:55:10,549 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50 empty. 2023-02-08 23:55:10,549 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf empty. 2023-02-08 23:55:10,550 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:10,550 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:10,550 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:10,550 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-08 23:55:10,647 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,647 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 23:55:10,848 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:10,848 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:10,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 23:55:10,957 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:10,958 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => bf85808cb96a3fd70743466476b776c5, NAME => 'testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:55:10,959 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => c0432a17915eb080602d94742abd61bf, NAME => 'testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:55:10,959 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => 83e0134f0861e917b7a0acc88b520e50, NAME => 'testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing 83e0134f0861e917b7a0acc88b520e50, disabling compactions & flushes 2023-02-08 23:55:10,967 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. after waiting 0 ms 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:10,967 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for 83e0134f0861e917b7a0acc88b520e50: Waiting for close lock at 1675900510967Disabling compacts and flushes for region at 1675900510967Disabling writes for close at 1675900510967Writing region close event to WAL at 1675900510967Closed at 1675900510967 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing c0432a17915eb080602d94742abd61bf, disabling compactions & flushes 2023-02-08 23:55:10,967 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. after waiting 0 ms 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:10,967 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:10,967 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for c0432a17915eb080602d94742abd61bf: Waiting for close lock at 1675900510967Disabling compacts and flushes for region at 1675900510967Disabling writes for close at 1675900510967Writing region close event to WAL at 1675900510967Closed at 1675900510967 2023-02-08 23:55:11,149 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:11,150 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:11,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 23:55:11,366 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:11,366 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing bf85808cb96a3fd70743466476b776c5, disabling compactions & flushes 2023-02-08 23:55:11,366 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:11,366 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:11,366 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. after waiting 0 ms 2023-02-08 23:55:11,366 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:11,366 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:11,366 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for bf85808cb96a3fd70743466476b776c5: Waiting for close lock at 1675900511366Disabling compacts and flushes for region at 1675900511366Disabling writes for close at 1675900511366Writing region close event to WAL at 1675900511366Closed at 1675900511366 2023-02-08 23:55:11,368 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:55:11,369 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900511369"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900511369"}]},"ts":"1675900511369"} 2023-02-08 23:55:11,369 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900511369"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900511369"}]},"ts":"1675900511369"} 2023-02-08 23:55:11,369 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900511369"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900511369"}]},"ts":"1675900511369"} 2023-02-08 23:55:11,375 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 23:55:11,375 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:55:11,375 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900511375"}]},"ts":"1675900511375"} 2023-02-08 23:55:11,376 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-08 23:55:11,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, ASSIGN}] 2023-02-08 23:55:11,382 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, ASSIGN 2023-02-08 23:55:11,382 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, ASSIGN 2023-02-08 23:55:11,382 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, ASSIGN 2023-02-08 23:55:11,383 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:55:11,383 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:55:11,383 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:55:11,534 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=c0432a17915eb080602d94742abd61bf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:11,534 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=83e0134f0861e917b7a0acc88b520e50, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:11,534 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bf85808cb96a3fd70743466476b776c5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:11,535 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900511534"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900511534"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900511534"}]},"ts":"1675900511534"} 2023-02-08 23:55:11,535 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900511534"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900511534"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900511534"}]},"ts":"1675900511534"} 2023-02-08 23:55:11,535 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900511534"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900511534"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900511534"}]},"ts":"1675900511534"} 2023-02-08 23:55:11,536 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure c0432a17915eb080602d94742abd61bf, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:11,537 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure 83e0134f0861e917b7a0acc88b520e50, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:11,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure bf85808cb96a3fd70743466476b776c5, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:11,651 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:11,651 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:11,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 23:55:11,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:11,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c0432a17915eb080602d94742abd61bf, NAME => 'testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 23:55:11,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:11,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:11,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:11,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:11,692 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:11,693 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1 2023-02-08 23:55:11,693 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1 2023-02-08 23:55:11,693 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0432a17915eb080602d94742abd61bf columnFamilyName family1 2023-02-08 23:55:11,694 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(310): Store=c0432a17915eb080602d94742abd61bf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:11,695 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:11,695 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2 2023-02-08 23:55:11,695 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2 2023-02-08 23:55:11,696 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0432a17915eb080602d94742abd61bf columnFamilyName family2 2023-02-08 23:55:11,697 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(310): Store=c0432a17915eb080602d94742abd61bf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:11,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:11,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:11,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:11,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:11,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:11,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c0432a17915eb080602d94742abd61bf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74816873, jitterRate=0.11485828459262848}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:11,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c0432a17915eb080602d94742abd61bf: Running coprocessor pre-open hook at 1675900511691Writing region info on filesystem at 1675900511691Initializing all the Stores at 1675900511691Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900511691Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900511692 (+1 ms)Cleaning up temporary data from old regions at 1675900511699 (+7 ms)Cleaning up detritus from prior splits at 1675900511699Running coprocessor post-open hooks at 1675900511703 (+4 ms)Region opened successfully at 1675900511703 2023-02-08 23:55:11,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf., pid=83, masterSystemTime=1675900511688 2023-02-08 23:55:11,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:11,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:11,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:11,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bf85808cb96a3fd70743466476b776c5, NAME => 'testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 23:55:11,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:11,705 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=c0432a17915eb080602d94742abd61bf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:11,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:11,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:11,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:11,705 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900511705"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900511705"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900511705"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900511705"}]},"ts":"1675900511705"} 2023-02-08 23:55:11,706 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:11,707 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1 2023-02-08 23:55:11,708 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1 2023-02-08 23:55:11,708 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-02-08 23:55:11,708 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure c0432a17915eb080602d94742abd61bf, server=jenkins-hbase4.apache.org,42703,1675900414768 in 170 msec 2023-02-08 23:55:11,708 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf85808cb96a3fd70743466476b776c5 columnFamilyName family1 2023-02-08 23:55:11,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, ASSIGN in 327 msec 2023-02-08 23:55:11,709 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(310): Store=bf85808cb96a3fd70743466476b776c5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:11,709 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:11,710 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2 2023-02-08 23:55:11,710 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2 2023-02-08 23:55:11,711 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf85808cb96a3fd70743466476b776c5 columnFamilyName family2 2023-02-08 23:55:11,712 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(310): Store=bf85808cb96a3fd70743466476b776c5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:11,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:11,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:11,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:11,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:11,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:11,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bf85808cb96a3fd70743466476b776c5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65149608, jitterRate=-0.029195189476013184}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:11,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bf85808cb96a3fd70743466476b776c5: Running coprocessor pre-open hook at 1675900511705Writing region info on filesystem at 1675900511705Initializing all the Stores at 1675900511706 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900511706Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900511706Cleaning up temporary data from old regions at 1675900511713 (+7 ms)Cleaning up detritus from prior splits at 1675900511714 (+1 ms)Running coprocessor post-open hooks at 1675900511717 (+3 ms)Region opened successfully at 1675900511717 2023-02-08 23:55:11,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5., pid=85, masterSystemTime=1675900511688 2023-02-08 23:55:11,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:11,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:11,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:11,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 83e0134f0861e917b7a0acc88b520e50, NAME => 'testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 23:55:11,719 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bf85808cb96a3fd70743466476b776c5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:11,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:11,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:11,719 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900511719"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900511719"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900511719"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900511719"}]},"ts":"1675900511719"} 2023-02-08 23:55:11,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:11,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:11,720 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:11,722 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1 2023-02-08 23:55:11,722 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1 2023-02-08 23:55:11,722 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=80 2023-02-08 23:55:11,722 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure bf85808cb96a3fd70743466476b776c5, server=jenkins-hbase4.apache.org,42703,1675900414768 in 183 msec 2023-02-08 23:55:11,722 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 83e0134f0861e917b7a0acc88b520e50 columnFamilyName family1 2023-02-08 23:55:11,723 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(310): Store=83e0134f0861e917b7a0acc88b520e50/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:11,723 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, ASSIGN in 342 msec 2023-02-08 23:55:11,723 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:11,724 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2 2023-02-08 23:55:11,724 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2 2023-02-08 23:55:11,724 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 83e0134f0861e917b7a0acc88b520e50 columnFamilyName family2 2023-02-08 23:55:11,725 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(310): Store=83e0134f0861e917b7a0acc88b520e50/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:11,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:11,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:11,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:11,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:11,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:11,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 83e0134f0861e917b7a0acc88b520e50; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66506992, jitterRate=-0.008968591690063477}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:11,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 83e0134f0861e917b7a0acc88b520e50: Running coprocessor pre-open hook at 1675900511719Writing region info on filesystem at 1675900511719Initializing all the Stores at 1675900511720 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900511720Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900511720Cleaning up temporary data from old regions at 1675900511726 (+6 ms)Cleaning up detritus from prior splits at 1675900511727 (+1 ms)Running coprocessor post-open hooks at 1675900511730 (+3 ms)Region opened successfully at 1675900511730 2023-02-08 23:55:11,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50., pid=84, masterSystemTime=1675900511688 2023-02-08 23:55:11,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:11,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:11,732 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=83e0134f0861e917b7a0acc88b520e50, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:11,732 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900511732"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900511732"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900511732"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900511732"}]},"ts":"1675900511732"} 2023-02-08 23:55:11,734 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-02-08 23:55:11,734 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure 83e0134f0861e917b7a0acc88b520e50, server=jenkins-hbase4.apache.org,42703,1675900414768 in 196 msec 2023-02-08 23:55:11,735 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-08 23:55:11,735 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, ASSIGN in 354 msec 2023-02-08 23:55:11,736 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:55:11,736 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900511736"}]},"ts":"1675900511736"} 2023-02-08 23:55:11,737 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-08 23:55:11,739 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:55:11,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.1950 sec 2023-02-08 23:55:11,930 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-08 23:55:12,652 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:12,652 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:12,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 23:55:12,653 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-08 23:55:12,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:12,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:12,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:12,664 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:12,664 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:12,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:12,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 23:55:12,666 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 23:55:12,666 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:12,666 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 23:55:12,666 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 23:55:12,667 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:12,667 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:12,668 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:12,668 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:12,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:12,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:12,668 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:12,668 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:12,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:12,668 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:12,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:12,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:12,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 23:55:12,669 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 23:55:12,669 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 23:55:12,669 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:12,669 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 23:55:12,669 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:12,669 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. started... 2023-02-08 23:55:12,673 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:12,673 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:12,673 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:12,673 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:12,673 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. started... 2023-02-08 23:55:12,673 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. started... 2023-02-08 23:55:12,673 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 83e0134f0861e917b7a0acc88b520e50 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 23:55:12,673 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c0432a17915eb080602d94742abd61bf 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 23:55:12,674 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bf85808cb96a3fd70743466476b776c5 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 23:55:12,692 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208eee36449625442d0980dcaeb5c877f3f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208eee36449625442d0980dcaeb5c877f3f 2023-02-08 23:55:13,087 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302084298f8536b12414885004ca6363d5751 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302084298f8536b12414885004ca6363d5751 2023-02-08 23:55:13,088 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020845886db6ce904d17bca6173c2e44fa1c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020845886db6ce904d17bca6173c2e44fa1c 2023-02-08 23:55:13,097 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/efc80c1c81864c3990e1fec54dc689c0 2023-02-08 23:55:13,498 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/be9073e70f064a84b2b0f1854b8db204 2023-02-08 23:55:13,498 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/0e38670639294b138a863ec328b3514c 2023-02-08 23:55:13,511 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020863c747205e36467b845e75e62af56bbc to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020863c747205e36467b845e75e62af56bbc 2023-02-08 23:55:13,913 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208cfacfd5ce0334e21ac2a185385ca0c86 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfacfd5ce0334e21ac2a185385ca0c86 2023-02-08 23:55:13,913 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208eeeb89abf9c5490299cfb4b07b224efd to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208eeeb89abf9c5490299cfb4b07b224efd 2023-02-08 23:55:13,916 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/4461f76f0e1a4a289ede864a928e7ef2 2023-02-08 23:55:13,920 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/efc80c1c81864c3990e1fec54dc689c0 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/efc80c1c81864c3990e1fec54dc689c0 2023-02-08 23:55:13,923 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/efc80c1c81864c3990e1fec54dc689c0, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 23:55:13,923 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/4461f76f0e1a4a289ede864a928e7ef2 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/4461f76f0e1a4a289ede864a928e7ef2 2023-02-08 23:55:13,926 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/4461f76f0e1a4a289ede864a928e7ef2, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 23:55:13,927 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 83e0134f0861e917b7a0acc88b520e50 in 1254ms, sequenceid=5, compaction requested=false 2023-02-08 23:55:13,928 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:13,928 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:13,928 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:14,318 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/f6702cb56c6049bdb1dcc89a1b85895d 2023-02-08 23:55:14,318 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/2d7243117fd7456eaf23062495841f44 2023-02-08 23:55:14,322 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/0e38670639294b138a863ec328b3514c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0e38670639294b138a863ec328b3514c 2023-02-08 23:55:14,322 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/be9073e70f064a84b2b0f1854b8db204 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/be9073e70f064a84b2b0f1854b8db204 2023-02-08 23:55:14,326 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/be9073e70f064a84b2b0f1854b8db204, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 23:55:14,326 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0e38670639294b138a863ec328b3514c, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 23:55:14,327 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/f6702cb56c6049bdb1dcc89a1b85895d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/f6702cb56c6049bdb1dcc89a1b85895d 2023-02-08 23:55:14,327 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/2d7243117fd7456eaf23062495841f44 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/2d7243117fd7456eaf23062495841f44 2023-02-08 23:55:14,330 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/f6702cb56c6049bdb1dcc89a1b85895d, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 23:55:14,330 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/2d7243117fd7456eaf23062495841f44, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 23:55:14,331 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for bf85808cb96a3fd70743466476b776c5 in 1657ms, sequenceid=5, compaction requested=false 2023-02-08 23:55:14,331 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:14,331 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:14,331 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:14,331 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for c0432a17915eb080602d94742abd61bf in 1658ms, sequenceid=5, compaction requested=false 2023-02-08 23:55:14,331 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:14,331 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:14,331 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:14,331 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:14,331 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,331 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 23:55:14,331 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 23:55:14,333 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:14,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:14,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:14,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:14,333 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:14,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:14,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:14,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:14,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:14,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 23:55:14,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45a77a6e[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:14,334 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 23:55:14,335 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:14,337 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:14,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:14,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:14,337 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 23:55:14,337 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 23:55:14,337 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:14,337 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,337 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:14,338 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,338 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 23:55:14,338 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:14,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:14,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:14,339 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 23:55:14,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:14,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:14,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:14,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:14,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:14,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:14,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 23:55:14,341 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 23:55:14,341 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:14,341 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:14,341 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 23:55:14,341 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:14,342 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:14,342 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:14,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:14,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:14,342 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:14,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:14,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:14,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:14,343 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:14,344 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:14,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:14,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:14,346 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-08 23:55:14,346 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 23:55:14,346 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:14,346 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:14,346 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:14,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:14,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:14,446 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:14,447 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:14,447 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:14,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:14,448 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:14,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:14,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:14,451 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:14,451 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:14,451 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:14,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:14,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 23:55:14,453 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 23:55:14,453 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:14,453 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 23:55:14,453 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 23:55:14,454 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:14,454 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:14,456 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,456 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:14,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:14,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:14,456 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:14,456 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:14,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:14,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:14,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:14,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:14,457 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 23:55:14,457 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-08 23:55:14,457 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 23:55:14,457 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 23:55:14,457 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:14,457 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 23:55:14,457 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:14,457 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:14,457 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:14,458 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:14,458 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. started... 2023-02-08 23:55:14,457 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. started... 2023-02-08 23:55:14,458 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. started... 2023-02-08 23:55:14,457 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:14,458 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 83e0134f0861e917b7a0acc88b520e50 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 23:55:14,458 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bf85808cb96a3fd70743466476b776c5 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 23:55:14,458 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c0432a17915eb080602d94742abd61bf 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 23:55:14,472 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302088edfd6d65d9c4a9c9ada8e78ee01fb64 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088edfd6d65d9c4a9c9ada8e78ee01fb64 2023-02-08 23:55:14,472 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020824290db562504ae099dc7ac42a91428a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020824290db562504ae099dc7ac42a91428a 2023-02-08 23:55:14,481 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302086879d9029967492f881893b3ece54a05 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302086879d9029967492f881893b3ece54a05 2023-02-08 23:55:14,883 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/ff75152bef214b51bf249e64e0341f34 2023-02-08 23:55:14,886 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/904a84fc1b8442109aa95d92a5281578 2023-02-08 23:55:14,891 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/8e816cbaac864e4bac1ad5c1403b2cda 2023-02-08 23:55:14,898 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208cfa91b29aad549c0ad57bb2dc1be2404 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfa91b29aad549c0ad57bb2dc1be2404 2023-02-08 23:55:15,300 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302089eda5bf68c95405c952b826fa83c8e5a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302089eda5bf68c95405c952b826fa83c8e5a 2023-02-08 23:55:15,308 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/5e0aadad3d8e4d40a9a856d5d0e9935d 2023-02-08 23:55:15,311 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020893dba873f1e74ce5afc7e102e2fbf73d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020893dba873f1e74ce5afc7e102e2fbf73d 2023-02-08 23:55:15,312 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/ff75152bef214b51bf249e64e0341f34 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ff75152bef214b51bf249e64e0341f34 2023-02-08 23:55:15,315 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ff75152bef214b51bf249e64e0341f34, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 23:55:15,316 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/5e0aadad3d8e4d40a9a856d5d0e9935d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/5e0aadad3d8e4d40a9a856d5d0e9935d 2023-02-08 23:55:15,319 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/5e0aadad3d8e4d40a9a856d5d0e9935d, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 23:55:15,320 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for bf85808cb96a3fd70743466476b776c5 in 862ms, sequenceid=9, compaction requested=false 2023-02-08 23:55:15,320 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:15,320 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:15,320 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:15,706 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/596a8c7e9ee74ee9a9c70737892c7b88 2023-02-08 23:55:15,709 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/904a84fc1b8442109aa95d92a5281578 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/904a84fc1b8442109aa95d92a5281578 2023-02-08 23:55:15,713 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/904a84fc1b8442109aa95d92a5281578, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 23:55:15,714 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/596a8c7e9ee74ee9a9c70737892c7b88 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/596a8c7e9ee74ee9a9c70737892c7b88 2023-02-08 23:55:15,716 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/7a764657617342fe908e99ce225c7881 2023-02-08 23:55:15,717 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/596a8c7e9ee74ee9a9c70737892c7b88, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 23:55:15,718 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for c0432a17915eb080602d94742abd61bf in 1260ms, sequenceid=9, compaction requested=false 2023-02-08 23:55:15,718 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:15,718 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:15,718 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:15,720 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/8e816cbaac864e4bac1ad5c1403b2cda as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/8e816cbaac864e4bac1ad5c1403b2cda 2023-02-08 23:55:15,723 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/8e816cbaac864e4bac1ad5c1403b2cda, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 23:55:15,723 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/7a764657617342fe908e99ce225c7881 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7a764657617342fe908e99ce225c7881 2023-02-08 23:55:15,726 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7a764657617342fe908e99ce225c7881, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 23:55:15,727 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 83e0134f0861e917b7a0acc88b520e50 in 1269ms, sequenceid=9, compaction requested=false 2023-02-08 23:55:15,727 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:15,727 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:15,727 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:15,727 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:15,727 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,727 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 23:55:15,727 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 23:55:15,729 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:15,729 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:15,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:15,729 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:15,729 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:15,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:15,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:15,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:15,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:15,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 23:55:15,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63fae3b8[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:15,730 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 23:55:15,730 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:15,731 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:15,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:15,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:15,731 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 23:55:15,731 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 23:55:15,731 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:15,731 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,731 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:15,732 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 23:55:15,732 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,732 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:15,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:15,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:15,733 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 23:55:15,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:15,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:15,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:15,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:15,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:15,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:15,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 23:55:15,735 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 23:55:15,735 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:15,735 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:15,735 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 23:55:15,735 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:15,738 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:15,738 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:15,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:15,738 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:15,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:15,738 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:15,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:15,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:15,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:15,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:15,739 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:15,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:15,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,741 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,741 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:15,741 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:15,741 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:15,742 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-08 23:55:15,741 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:15,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 23:55:15,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:15,741 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:15,742 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:15,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:15,742 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:15,742 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:15,742 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:15,742 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:15,742 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:15,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:15,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:15,767 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:55:15,842 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:15,842 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:15,842 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:15,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:15,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:15,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:15,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:15,847 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:15,847 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:15,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:15,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:15,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 23:55:15,849 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 23:55:15,849 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:15,849 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 23:55:15,849 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 23:55:15,850 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:15,850 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:15,851 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,851 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:15,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:15,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:15,851 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:15,851 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:15,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:15,852 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:15,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:15,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:15,852 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 23:55:15,852 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-08 23:55:15,852 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 23:55:15,852 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 23:55:15,852 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:15,852 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 23:55:15,852 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:15,853 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. started... 2023-02-08 23:55:15,853 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:15,853 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:15,853 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 83e0134f0861e917b7a0acc88b520e50 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 23:55:15,853 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:15,853 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:15,853 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. started... 2023-02-08 23:55:15,853 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. started... 2023-02-08 23:55:15,853 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bf85808cb96a3fd70743466476b776c5 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 23:55:15,853 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c0432a17915eb080602d94742abd61bf 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 23:55:16,269 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302088eb3af42556a4def9b2293f1392ac0a3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088eb3af42556a4def9b2293f1392ac0a3 2023-02-08 23:55:16,269 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302082fa734f47d4b4337ace2dfaa02fb8c4a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302082fa734f47d4b4337ace2dfaa02fb8c4a 2023-02-08 23:55:16,269 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302084c94eb80abf24e1f920601f01f56cda5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302084c94eb80abf24e1f920601f01f56cda5 2023-02-08 23:55:16,679 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/97d9beddfbca474d9f178711cadcf3bd 2023-02-08 23:55:16,679 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/388d3ec4cc4041828e2fe571247f5aad 2023-02-08 23:55:16,679 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/41e307b1667b4e89bec738267d5e3eb7 2023-02-08 23:55:16,693 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208539c497a44e34437b0679f61feaab580 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208539c497a44e34437b0679f61feaab580 2023-02-08 23:55:16,693 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208e2df64581a9744a19609b126756323be to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208e2df64581a9744a19609b126756323be 2023-02-08 23:55:17,093 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208fb345cb5d4514f6284d52b663db4ff57 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208fb345cb5d4514f6284d52b663db4ff57 2023-02-08 23:55:17,099 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/73bfac9ee9854508b7540975ac4e8772 2023-02-08 23:55:17,099 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/c6f4612529414554b9acfc9c01140bb5 2023-02-08 23:55:17,103 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/388d3ec4cc4041828e2fe571247f5aad as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/388d3ec4cc4041828e2fe571247f5aad 2023-02-08 23:55:17,103 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/97d9beddfbca474d9f178711cadcf3bd as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/97d9beddfbca474d9f178711cadcf3bd 2023-02-08 23:55:17,107 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/97d9beddfbca474d9f178711cadcf3bd, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 23:55:17,107 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/388d3ec4cc4041828e2fe571247f5aad, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 23:55:17,108 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/73bfac9ee9854508b7540975ac4e8772 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/73bfac9ee9854508b7540975ac4e8772 2023-02-08 23:55:17,108 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/c6f4612529414554b9acfc9c01140bb5 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/c6f4612529414554b9acfc9c01140bb5 2023-02-08 23:55:17,112 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/73bfac9ee9854508b7540975ac4e8772, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 23:55:17,112 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/c6f4612529414554b9acfc9c01140bb5, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 23:55:17,113 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for c0432a17915eb080602d94742abd61bf in 1260ms, sequenceid=13, compaction requested=true 2023-02-08 23:55:17,113 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:17,113 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:17,113 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:17,113 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for bf85808cb96a3fd70743466476b776c5 in 1260ms, sequenceid=13, compaction requested=true 2023-02-08 23:55:17,113 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:17,113 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:17,113 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:17,498 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/7599740811c241538c4b650f4a4a7446 2023-02-08 23:55:17,503 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/41e307b1667b4e89bec738267d5e3eb7 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/41e307b1667b4e89bec738267d5e3eb7 2023-02-08 23:55:17,507 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/41e307b1667b4e89bec738267d5e3eb7, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 23:55:17,507 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/7599740811c241538c4b650f4a4a7446 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7599740811c241538c4b650f4a4a7446 2023-02-08 23:55:17,511 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7599740811c241538c4b650f4a4a7446, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 23:55:17,512 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 83e0134f0861e917b7a0acc88b520e50 in 1659ms, sequenceid=13, compaction requested=true 2023-02-08 23:55:17,512 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:17,512 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 23:55:17,512 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 23:55:17,513 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,514 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:17,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:17,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:17,514 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:17,514 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:17,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:17,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:17,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:17,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:17,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 23:55:17,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@130e4443[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:17,515 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 23:55:17,515 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:17,517 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:17,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:17,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:17,518 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,518 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 23:55:17,518 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 23:55:17,518 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:17,518 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:17,519 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 23:55:17,519 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,519 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:17,519 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 23:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:17,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:17,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 23:55:17,521 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 23:55:17,521 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:17,521 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:17,521 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 23:55:17,521 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:17,524 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:17,524 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:17,525 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:17,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:17,525 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:17,526 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:17,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,528 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,528 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:17,528 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:17,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:17,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 23:55:17,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:17,528 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:17,528 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:17,528 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:17,528 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:17,528 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:17,529 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:17,529 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:17,529 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:17,529 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:17,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:17,629 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:17,629 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:17,629 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:17,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:17,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:17,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:17,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:17,633 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:17,633 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:17,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:17,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:17,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 23:55:17,635 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 23:55:17,635 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:17,635 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 23:55:17,635 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 23:55:17,636 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:17,636 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:17,637 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,637 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:17,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:17,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:17,637 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:17,637 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:17,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:17,638 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:17,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:17,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:17,638 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 23:55:17,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-08 23:55:17,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 23:55:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 23:55:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:17,638 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 23:55:17,638 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:17,639 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. started... 2023-02-08 23:55:17,639 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:17,639 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:17,639 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:17,639 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. started... 2023-02-08 23:55:17,639 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:17,639 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. started... 2023-02-08 23:55:17,639 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c0432a17915eb080602d94742abd61bf 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 23:55:17,639 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 83e0134f0861e917b7a0acc88b520e50 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 23:55:17,639 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bf85808cb96a3fd70743466476b776c5 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 23:55:18,055 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208b67eff96efab410aa112021b51ca7dfc to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208b67eff96efab410aa112021b51ca7dfc 2023-02-08 23:55:18,056 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020896ce51894cc442cbaafeb262785520c6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020896ce51894cc442cbaafeb262785520c6 2023-02-08 23:55:18,056 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302080854ba64b08e4a44a8ef84237326c925 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302080854ba64b08e4a44a8ef84237326c925 2023-02-08 23:55:18,465 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/9589f8bc35fd4c6e98792433f593e54c 2023-02-08 23:55:18,466 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/b18bc41b32274e03a92f7d9f409420b9 2023-02-08 23:55:18,470 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/d29e85ef6e604a50a4aa2d61a274e51f 2023-02-08 23:55:18,483 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302083870c496f39e4adead682c1808ae9f03 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302083870c496f39e4adead682c1808ae9f03 2023-02-08 23:55:18,881 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208c4561f861fb945cc896cdeb3f74c35e1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208c4561f861fb945cc896cdeb3f74c35e1 2023-02-08 23:55:18,892 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020826e765438cb541d8b814ee8f7e8168ac to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020826e765438cb541d8b814ee8f7e8168ac 2023-02-08 23:55:18,893 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb 2023-02-08 23:55:18,897 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/9589f8bc35fd4c6e98792433f593e54c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/9589f8bc35fd4c6e98792433f593e54c 2023-02-08 23:55:18,901 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/9589f8bc35fd4c6e98792433f593e54c, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 23:55:18,901 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb 2023-02-08 23:55:18,904 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 23:55:18,905 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 83e0134f0861e917b7a0acc88b520e50 in 1266ms, sequenceid=17, compaction requested=true 2023-02-08 23:55:18,905 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:18,905 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:18,905 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:19,286 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/20b322639109490b8cafc695fbc09c2f 2023-02-08 23:55:19,290 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/b18bc41b32274e03a92f7d9f409420b9 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/b18bc41b32274e03a92f7d9f409420b9 2023-02-08 23:55:19,293 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/b18bc41b32274e03a92f7d9f409420b9, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 23:55:19,294 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/20b322639109490b8cafc695fbc09c2f as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/20b322639109490b8cafc695fbc09c2f 2023-02-08 23:55:19,296 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/0725b9bf527c498bb05cd7b12e41b68a 2023-02-08 23:55:19,297 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/20b322639109490b8cafc695fbc09c2f, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 23:55:19,298 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for bf85808cb96a3fd70743466476b776c5 in 1659ms, sequenceid=17, compaction requested=true 2023-02-08 23:55:19,298 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:19,298 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:19,298 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:19,300 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/d29e85ef6e604a50a4aa2d61a274e51f as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/d29e85ef6e604a50a4aa2d61a274e51f 2023-02-08 23:55:19,304 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/d29e85ef6e604a50a4aa2d61a274e51f, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 23:55:19,304 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/0725b9bf527c498bb05cd7b12e41b68a as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/0725b9bf527c498bb05cd7b12e41b68a 2023-02-08 23:55:19,308 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/0725b9bf527c498bb05cd7b12e41b68a, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 23:55:19,309 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for c0432a17915eb080602d94742abd61bf in 1670ms, sequenceid=17, compaction requested=true 2023-02-08 23:55:19,309 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:19,309 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:19,309 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:19,309 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:19,309 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,309 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 23:55:19,309 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 23:55:19,312 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:19,312 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:19,312 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:19,312 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:19,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:19,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 23:55:19,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@576b632f[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:19,313 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 23:55:19,313 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:19,314 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:19,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:19,315 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 23:55:19,315 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 23:55:19,315 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:19,315 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,315 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:19,316 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 23:55:19,316 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,316 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:19,316 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 23:55:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:19,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:19,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:19,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:19,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:19,318 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 23:55:19,318 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:19,318 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:19,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 23:55:19,318 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 23:55:19,318 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:19,319 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:19,319 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:19,319 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:19,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:19,320 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:19,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:19,321 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:19,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:19,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,323 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,323 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:19,323 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:19,323 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:19,324 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:19,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 23:55:19,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:19,324 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:19,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:19,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:19,324 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,324 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:19,324 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:19,324 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:19,324 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:19,324 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:19,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:19,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:19,424 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:19,424 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:19,424 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:19,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:19,434 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:19,434 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:19,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 23:55:19,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675900519435 (current time:1675900519436). 2023-02-08 23:55:19,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-08 23:55:19,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] zookeeper.ReadOnlyZKClient(139): Connect 0x0d00adc3 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:55:19,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d5cb70b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:55:19,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:55:19,449 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58498, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:55:19,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0d00adc3 to 127.0.0.1:54774 2023-02-08 23:55:19,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:19,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-08 23:55:19,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 23:55:19,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 23:55:19,460 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 23:55:19,461 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675901119461 2023-02-08 23:55:19,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 23:55:19,461 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1675900519425 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-08 23:55:19,462 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 23:55:19,463 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-08 23:55:19,463 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-02-08 23:55:19,464 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-08 23:55:19,464 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1675901119464 2023-02-08 23:55:19,466 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:19,466 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-08 23:55:19,566 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:19,567 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:19,567 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:19,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:19,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 23:55:19,571 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-08 23:55:19,771 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:19,771 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:19,771 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:19,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:19,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 23:55:19,772 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-08 23:55:19,872 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1675900519425 2023-02-08 23:55:19,872 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1675900519425' 2023-02-08 23:55:19,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-08 23:55:19,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1675900519425' starting 'acquire' 2023-02-08 23:55:19,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1675900519425', kicking off acquire phase on members. 2023-02-08 23:55:19,873 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675900519425 2023-02-08 23:55:19,873 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1675900519425 2023-02-08 23:55:19,874 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,874 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-08 23:55:19,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-08 23:55:19,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 23:55:19,874 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,874 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:19,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1675900519425 2023-02-08 23:55:19,875 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675900519425 2023-02-08 23:55:19,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-08 23:55:19,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1675900519425 2023-02-08 23:55:19,875 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1675900519425 from table testCompactionWithHFileLink type FLUSH 2023-02-08 23:55:19,877 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1675900519425 2023-02-08 23:55:19,877 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1675900519425' with timeout 300000ms 2023-02-08 23:55:19,877 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-08 23:55:19,877 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1675900519425' starting 'acquire' stage 2023-02-08 23:55:19,877 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1675900519425' locally acquired 2023-02-08 23:55:19,877 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (snaptb-1675900519425) in zk 2023-02-08 23:55:19,880 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1675900519425 2023-02-08 23:55:19,880 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:19,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-08 23:55:19,880 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675900519425 2023-02-08 23:55:19,880 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1675900519425' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:19,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:19,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:19,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675900519425 2023-02-08 23:55:19,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:19,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'snaptb-1675900519425' on coordinator 2023-02-08 23:55:19,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29e117a6[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:19,881 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1675900519425' starting 'in-barrier' execution. 2023-02-08 23:55:19,881 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1675900519425 2023-02-08 23:55:19,882 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675900519425 2023-02-08 23:55:19,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1675900519425 2023-02-08 23:55:19,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1675900519425 2023-02-08 23:55:19,882 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1675900519425' received 'reached' from coordinator. 2023-02-08 23:55:19,882 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:19,883 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:19,883 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:19,883 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-08 23:55:19,883 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:19,883 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-08 23:55:19,883 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:19,884 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. started... 2023-02-08 23:55:19,884 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. started... 2023-02-08 23:55:19,884 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. started... 2023-02-08 23:55:19,884 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:19,884 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:19,884 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:19,885 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.' region-info for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,885 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.' region-info for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,885 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.' region-info for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,890 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-08 23:55:19,890 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-08 23:55:19,890 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-08 23:55:19,893 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/be9073e70f064a84b2b0f1854b8db204, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ff75152bef214b51bf249e64e0341f34, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/97d9beddfbca474d9f178711cadcf3bd, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/b18bc41b32274e03a92f7d9f409420b9] hfiles 2023-02-08 23:55:19,893 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0e38670639294b138a863ec328b3514c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/904a84fc1b8442109aa95d92a5281578, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/388d3ec4cc4041828e2fe571247f5aad, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/d29e85ef6e604a50a4aa2d61a274e51f] hfiles 2023-02-08 23:55:19,893 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/efc80c1c81864c3990e1fec54dc689c0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/8e816cbaac864e4bac1ad5c1403b2cda, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/41e307b1667b4e89bec738267d5e3eb7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/9589f8bc35fd4c6e98792433f593e54c] hfiles 2023-02-08 23:55:19,894 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0e38670639294b138a863ec328b3514c for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,894 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/be9073e70f064a84b2b0f1854b8db204 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,894 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/efc80c1c81864c3990e1fec54dc689c0 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,897 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ff75152bef214b51bf249e64e0341f34 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,897 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/904a84fc1b8442109aa95d92a5281578 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,897 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/8e816cbaac864e4bac1ad5c1403b2cda for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,897 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/388d3ec4cc4041828e2fe571247f5aad for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,897 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/97d9beddfbca474d9f178711cadcf3bd for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,897 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/d29e85ef6e604a50a4aa2d61a274e51f for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,897 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/41e307b1667b4e89bec738267d5e3eb7 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,897 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/b18bc41b32274e03a92f7d9f409420b9 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/9589f8bc35fd4c6e98792433f593e54c for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/f6702cb56c6049bdb1dcc89a1b85895d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/5e0aadad3d8e4d40a9a856d5d0e9935d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/c6f4612529414554b9acfc9c01140bb5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/20b322639109490b8cafc695fbc09c2f] hfiles 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/f6702cb56c6049bdb1dcc89a1b85895d for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/4461f76f0e1a4a289ede864a928e7ef2, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7a764657617342fe908e99ce225c7881, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7599740811c241538c4b650f4a4a7446, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb] hfiles 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/5e0aadad3d8e4d40a9a856d5d0e9935d for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/2d7243117fd7456eaf23062495841f44, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/596a8c7e9ee74ee9a9c70737892c7b88, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/73bfac9ee9854508b7540975ac4e8772, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/0725b9bf527c498bb05cd7b12e41b68a] hfiles 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/c6f4612529414554b9acfc9c01140bb5 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/4461f76f0e1a4a289ede864a928e7ef2 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/20b322639109490b8cafc695fbc09c2f for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7a764657617342fe908e99ce225c7881 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/2d7243117fd7456eaf23062495841f44 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7599740811c241538c4b650f4a4a7446 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/596a8c7e9ee74ee9a9c70737892c7b88 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/73bfac9ee9854508b7540975ac4e8772 for snapshot=snaptb-1675900519425 2023-02-08 23:55:19,898 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/0725b9bf527c498bb05cd7b12e41b68a for snapshot=snaptb-1675900519425 2023-02-08 23:55:20,072 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:20,072 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:20,072 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:20,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:20,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 23:55:20,073 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-08 23:55:20,304 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. completed. 2023-02-08 23:55:20,304 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. completed. 2023-02-08 23:55:20,304 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:20,304 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. completed. 2023-02-08 23:55:20,304 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:20,304 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:20,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-08 23:55:20,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-08 23:55:20,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-08 23:55:20,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-08 23:55:20,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1675900519425' locally completed 2023-02-08 23:55:20,305 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1675900519425' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:20,306 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,306 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1675900519425' has notified controller of completion 2023-02-08 23:55:20,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:20,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-08 23:55:20,307 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:20,307 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1675900519425' completed. 2023-02-08 23:55:20,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:20,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:20,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675900519425 2023-02-08 23:55:20,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:20,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675900519425 2023-02-08 23:55:20,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1675900519425' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:20,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'snaptb-1675900519425', counting down latch. Waiting for 0 more 2023-02-08 23:55:20,309 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1675900519425' execution completed 2023-02-08 23:55:20,309 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:20,309 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:20,309 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1675900519425 2023-02-08 23:55:20,309 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1675900519425including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-08 23:55:20,310 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675900519425 2023-02-08 23:55:20,310 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675900519425 2023-02-08 23:55:20,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1675900519425 2023-02-08 23:55:20,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1675900519425 2023-02-08 23:55:20,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:20,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675900519425 2023-02-08 23:55:20,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-08 23:55:20,310 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-08 23:55:20,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-08 23:55:20,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:20,311 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 23:55:20,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675900519425 2023-02-08 23:55:20,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675900519425 2023-02-08 23:55:20,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:20,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675900519425 2023-02-08 23:55:20,312 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:20,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675900519425 2023-02-08 23:55:20,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,315 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,315 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-08 23:55:20,315 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675900519425 2023-02-08 23:55:20,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-08 23:55:20,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 23:55:20,315 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1675900519425 2023-02-08 23:55:20,315 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-08 23:55:20,315 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:20,315 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-08 23:55:20,315 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675900519425 2023-02-08 23:55:20,315 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675900519425/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:20,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1675900519425 because node does not exist (not an error) 2023-02-08 23:55:20,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-08 23:55:20,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 23:55:20,315 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1675900519425 2023-02-08 23:55:20,315 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675900519425 2023-02-08 23:55:20,316 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-08 23:55:20,316 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675900519425 2023-02-08 23:55:20,316 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675900519425 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302084c94eb80abf24e1f920601f01f56cda5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302086879d9029967492f881893b3ece54a05, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020896ce51894cc442cbaafeb262785520c6, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208eee36449625442d0980dcaeb5c877f3f, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302080854ba64b08e4a44a8ef84237326c925, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020824290db562504ae099dc7ac42a91428a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302082fa734f47d4b4337ace2dfaa02fb8c4a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302084298f8536b12414885004ca6363d5751, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020845886db6ce904d17bca6173c2e44fa1c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088eb3af42556a4def9b2293f1392ac0a3, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088edfd6d65d9c4a9c9ada8e78ee01fb64, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208b67eff96efab410aa112021b51ca7dfc] mob files 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302084c94eb80abf24e1f920601f01f56cda5 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302086879d9029967492f881893b3ece54a05 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020896ce51894cc442cbaafeb262785520c6 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208eee36449625442d0980dcaeb5c877f3f 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302080854ba64b08e4a44a8ef84237326c925 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020824290db562504ae099dc7ac42a91428a 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302082fa734f47d4b4337ace2dfaa02fb8c4a 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302084298f8536b12414885004ca6363d5751 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020845886db6ce904d17bca6173c2e44fa1c 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088eb3af42556a4def9b2293f1392ac0a3 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088edfd6d65d9c4a9c9ada8e78ee01fb64 2023-02-08 23:55:20,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208b67eff96efab410aa112021b51ca7dfc 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302083870c496f39e4adead682c1808ae9f03, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020863c747205e36467b845e75e62af56bbc, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020893dba873f1e74ce5afc7e102e2fbf73d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208fb345cb5d4514f6284d52b663db4ff57, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020826e765438cb541d8b814ee8f7e8168ac, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302089eda5bf68c95405c952b826fa83c8e5a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208e2df64581a9744a19609b126756323be, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208eeeb89abf9c5490299cfb4b07b224efd, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208539c497a44e34437b0679f61feaab580, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208c4561f861fb945cc896cdeb3f74c35e1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfa91b29aad549c0ad57bb2dc1be2404, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfacfd5ce0334e21ac2a185385ca0c86] mob files 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302083870c496f39e4adead682c1808ae9f03 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020863c747205e36467b845e75e62af56bbc 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020893dba873f1e74ce5afc7e102e2fbf73d 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208fb345cb5d4514f6284d52b663db4ff57 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020826e765438cb541d8b814ee8f7e8168ac 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302089eda5bf68c95405c952b826fa83c8e5a 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208e2df64581a9744a19609b126756323be 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208eeeb89abf9c5490299cfb4b07b224efd 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208539c497a44e34437b0679f61feaab580 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208c4561f861fb945cc896cdeb3f74c35e1 2023-02-08 23:55:20,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfa91b29aad549c0ad57bb2dc1be2404 2023-02-08 23:55:20,319 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfacfd5ce0334e21ac2a185385ca0c86 2023-02-08 23:55:20,573 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:20,573 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:20,573 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:20,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:20,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 23:55:20,574 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-08 23:55:20,723 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1675900519425 2023-02-08 23:55:20,725 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/.tmp/snaptb-1675900519425 2023-02-08 23:55:21,171 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/.tmp/snaptb-1675900519425 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/snaptb-1675900519425 2023-02-08 23:55:21,575 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:21,575 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:21,575 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:21,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:21,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 23:55:21,576 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-02-08 23:55:21,986 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1675900519425 of table testCompactionWithHFileLink completed 2023-02-08 23:55:21,986 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/.tmp/snaptb-1675900519425 2023-02-08 23:55:21,986 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/.tmp/snaptb-1675900519425 2023-02-08 23:55:21,986 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1675900519425 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675900519461Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1675900520723 (+1262 ms)Consolidate snapshot: snaptb-1675900519425 at 1675900520723Loading Region manifests for snaptb-1675900519425 at 1675900520724 (+1 ms)Writing data manifest for snaptb-1675900519425 at 1675900520736 (+12 ms)Verifying snapshot: snaptb-1675900519425 at 1675900521144 (+408 ms)Snapshot snaptb-1675900519425 of table testCompactionWithHFileLink completed at 1675900521986 (+842 ms) 2023-02-08 23:55:21,987 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-08 23:55:21,988 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.5250 sec 2023-02-08 23:55:23,576 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:23,576 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:23,576 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:23,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:23,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-08 23:55:23,604 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:23,604 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:23,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:23,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:23,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 23:55:23,606 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 23:55:23,606 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:23,606 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 23:55:23,607 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 23:55:23,607 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:23,607 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:23,609 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:23,609 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:23,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:23,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:23,610 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:23,610 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:23,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:23,610 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:23,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:23,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:23,610 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 23:55:23,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-08 23:55:23,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 23:55:23,611 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 23:55:23,611 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:23,611 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 23:55:23,611 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:23,611 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:23,611 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:23,611 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:23,611 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. started... 2023-02-08 23:55:23,611 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. started... 2023-02-08 23:55:23,611 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:23,611 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 83e0134f0861e917b7a0acc88b520e50 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 23:55:23,611 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing bf85808cb96a3fd70743466476b776c5 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 23:55:23,611 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. started... 2023-02-08 23:55:23,611 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c0432a17915eb080602d94742abd61bf 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 23:55:24,428 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/6c06424b927e49c88c7077e2a50605ec 2023-02-08 23:55:24,428 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/630524251a304b0f836d782eab670beb 2023-02-08 23:55:24,428 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/4b56a72a7133430a8a5ce3984c122b63 2023-02-08 23:55:24,432 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c06424b927e49c88c7077e2a50605ec 2023-02-08 23:55:24,432 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b56a72a7133430a8a5ce3984c122b63 2023-02-08 23:55:24,432 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 630524251a304b0f836d782eab670beb 2023-02-08 23:55:25,249 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/2c3fa0ceb2094c30b0c89a21903c55a6 2023-02-08 23:55:25,249 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/aa9ee19e33a5485c8247adf9007236ea 2023-02-08 23:55:25,253 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c3fa0ceb2094c30b0c89a21903c55a6 2023-02-08 23:55:25,253 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/aa53a1d627a14499bb9a101f0b9e23ea 2023-02-08 23:55:25,253 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa9ee19e33a5485c8247adf9007236ea 2023-02-08 23:55:25,253 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/4b56a72a7133430a8a5ce3984c122b63 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/4b56a72a7133430a8a5ce3984c122b63 2023-02-08 23:55:25,253 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/6c06424b927e49c88c7077e2a50605ec as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6c06424b927e49c88c7077e2a50605ec 2023-02-08 23:55:25,256 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa53a1d627a14499bb9a101f0b9e23ea 2023-02-08 23:55:25,257 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/630524251a304b0f836d782eab670beb as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/630524251a304b0f836d782eab670beb 2023-02-08 23:55:25,257 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b56a72a7133430a8a5ce3984c122b63 2023-02-08 23:55:25,257 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/4b56a72a7133430a8a5ce3984c122b63, entries=3, sequenceid=24, filesize=5.0 K 2023-02-08 23:55:25,257 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c06424b927e49c88c7077e2a50605ec 2023-02-08 23:55:25,257 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6c06424b927e49c88c7077e2a50605ec, entries=3, sequenceid=24, filesize=5.0 K 2023-02-08 23:55:25,258 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/2c3fa0ceb2094c30b0c89a21903c55a6 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/2c3fa0ceb2094c30b0c89a21903c55a6 2023-02-08 23:55:25,258 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/aa9ee19e33a5485c8247adf9007236ea as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/aa9ee19e33a5485c8247adf9007236ea 2023-02-08 23:55:25,260 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 630524251a304b0f836d782eab670beb 2023-02-08 23:55:25,260 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/630524251a304b0f836d782eab670beb, entries=3, sequenceid=24, filesize=5.0 K 2023-02-08 23:55:25,261 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/aa53a1d627a14499bb9a101f0b9e23ea as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/aa53a1d627a14499bb9a101f0b9e23ea 2023-02-08 23:55:25,262 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa9ee19e33a5485c8247adf9007236ea 2023-02-08 23:55:25,262 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2c3fa0ceb2094c30b0c89a21903c55a6 2023-02-08 23:55:25,262 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/aa9ee19e33a5485c8247adf9007236ea, entries=1, sequenceid=24, filesize=5.0 K 2023-02-08 23:55:25,262 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/2c3fa0ceb2094c30b0c89a21903c55a6, entries=1, sequenceid=24, filesize=5.0 K 2023-02-08 23:55:25,263 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for bf85808cb96a3fd70743466476b776c5 in 1652ms, sequenceid=24, compaction requested=true 2023-02-08 23:55:25,263 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:25,263 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:25,263 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 83e0134f0861e917b7a0acc88b520e50 in 1652ms, sequenceid=24, compaction requested=true 2023-02-08 23:55:25,263 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:25,263 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:25,263 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:25,263 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:25,267 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa53a1d627a14499bb9a101f0b9e23ea 2023-02-08 23:55:25,267 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/aa53a1d627a14499bb9a101f0b9e23ea, entries=1, sequenceid=24, filesize=5.0 K 2023-02-08 23:55:25,267 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for c0432a17915eb080602d94742abd61bf in 1656ms, sequenceid=24, compaction requested=true 2023-02-08 23:55:25,267 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:25,267 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:25,267 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:25,268 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:25,268 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,268 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 23:55:25,268 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 23:55:25,269 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:25,270 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:25,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:25,270 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:25,270 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:25,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:25,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:25,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:25,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:25,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 23:55:25,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1db2c601[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:25,271 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 23:55:25,271 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:25,273 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:25,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:25,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:25,273 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 23:55:25,273 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 23:55:25,273 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:25,273 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,273 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:25,278 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 23:55:25,278 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,278 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:25,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:25,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:25,278 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 23:55:25,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:25,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:25,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:25,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:25,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:25,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:25,281 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 23:55:25,281 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 23:55:25,281 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:25,281 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:25,281 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 23:55:25,281 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:25,282 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:25,282 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:25,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:25,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:25,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:25,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:25,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:25,282 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:25,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:25,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:25,283 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:25,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:25,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:25,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:25,284 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 23:55:25,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:25,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 23:55:25,286 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:25,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:25,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:25,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-08 23:55:25,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:25,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:25,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:25,286 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:25,387 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:25,387 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:25,387 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:25,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:25,388 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:25,388 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): 83e0134f0861e917b7a0acc88b520e50/family1 is initiating major compaction (all files) 2023-02-08 23:55:25,389 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:25,389 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:25,389 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of 83e0134f0861e917b7a0acc88b520e50/family1 in testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:25,389 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/efc80c1c81864c3990e1fec54dc689c0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/8e816cbaac864e4bac1ad5c1403b2cda, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/41e307b1667b4e89bec738267d5e3eb7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/9589f8bc35fd4c6e98792433f593e54c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/4b56a72a7133430a8a5ce3984c122b63] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp, totalSize=26.7 K 2023-02-08 23:55:25,389 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting efc80c1c81864c3990e1fec54dc689c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512661 2023-02-08 23:55:25,390 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 8e816cbaac864e4bac1ad5c1403b2cda, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514449 2023-02-08 23:55:25,390 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 41e307b1667b4e89bec738267d5e3eb7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515845 2023-02-08 23:55:25,390 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 9589f8bc35fd4c6e98792433f593e54c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:25,391 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 4b56a72a7133430a8a5ce3984c122b63, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 23:55:25,403 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): 83e0134f0861e917b7a0acc88b520e50#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:25,810 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302086fdac96439a746ee9630abc2ec957784_del 2023-02-08 23:55:25,811 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302086fdac96439a746ee9630abc2ec957784_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302086fdac96439a746ee9630abc2ec957784_del 2023-02-08 23:55:26,623 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69c7c8ca5ef14025be53d442401acd49 2023-02-08 23:55:26,624 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/69c7c8ca5ef14025be53d442401acd49 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/69c7c8ca5ef14025be53d442401acd49 2023-02-08 23:55:26,628 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69c7c8ca5ef14025be53d442401acd49 2023-02-08 23:55:26,628 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 83e0134f0861e917b7a0acc88b520e50/family1 of 83e0134f0861e917b7a0acc88b520e50 into 69c7c8ca5ef14025be53d442401acd49(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:26,629 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:26,629 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:26,629 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): 83e0134f0861e917b7a0acc88b520e50/family2 is initiating major compaction (all files) 2023-02-08 23:55:26,629 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:26,629 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:26,630 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of 83e0134f0861e917b7a0acc88b520e50/family2 in testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:26,630 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/4461f76f0e1a4a289ede864a928e7ef2, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7a764657617342fe908e99ce225c7881, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7599740811c241538c4b650f4a4a7446, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/2c3fa0ceb2094c30b0c89a21903c55a6] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp, totalSize=25.4 K 2023-02-08 23:55:26,630 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 4461f76f0e1a4a289ede864a928e7ef2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512661 2023-02-08 23:55:26,630 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 7a764657617342fe908e99ce225c7881, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514449 2023-02-08 23:55:26,631 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 7599740811c241538c4b650f4a4a7446, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515845 2023-02-08 23:55:26,631 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 5fa5faab97ca4ee48dc3d414eaa6e9bb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:26,631 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 2c3fa0ceb2094c30b0c89a21903c55a6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 23:55:26,643 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): 83e0134f0861e917b7a0acc88b520e50#family2#compaction#108 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:26,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/efc80c1c81864c3990e1fec54dc689c0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/8e816cbaac864e4bac1ad5c1403b2cda, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/41e307b1667b4e89bec738267d5e3eb7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/9589f8bc35fd4c6e98792433f593e54c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/4b56a72a7133430a8a5ce3984c122b63] to archive 2023-02-08 23:55:26,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:26,658 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/efc80c1c81864c3990e1fec54dc689c0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/efc80c1c81864c3990e1fec54dc689c0 2023-02-08 23:55:26,659 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/8e816cbaac864e4bac1ad5c1403b2cda to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/8e816cbaac864e4bac1ad5c1403b2cda 2023-02-08 23:55:26,660 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/41e307b1667b4e89bec738267d5e3eb7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/41e307b1667b4e89bec738267d5e3eb7 2023-02-08 23:55:26,660 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/9589f8bc35fd4c6e98792433f593e54c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/9589f8bc35fd4c6e98792433f593e54c 2023-02-08 23:55:26,661 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/4b56a72a7133430a8a5ce3984c122b63 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/4b56a72a7133430a8a5ce3984c122b63 2023-02-08 23:55:27,055 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230208c6b4d5234e7a421098a7c9f75c24ab74_del 2023-02-08 23:55:27,055 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208c6b4d5234e7a421098a7c9f75c24ab74_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208c6b4d5234e7a421098a7c9f75c24ab74_del 2023-02-08 23:55:27,869 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 15d177a2fbf0481587bafc06b15c6690 2023-02-08 23:55:27,870 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/15d177a2fbf0481587bafc06b15c6690 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/15d177a2fbf0481587bafc06b15c6690 2023-02-08 23:55:27,873 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 15d177a2fbf0481587bafc06b15c6690 2023-02-08 23:55:27,874 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 83e0134f0861e917b7a0acc88b520e50/family2 of 83e0134f0861e917b7a0acc88b520e50 into 15d177a2fbf0481587bafc06b15c6690(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:27,874 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:27,874 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:27,874 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): bf85808cb96a3fd70743466476b776c5/family1 is initiating major compaction (all files) 2023-02-08 23:55:27,875 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:27,875 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:27,876 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of bf85808cb96a3fd70743466476b776c5/family1 in testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:27,876 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/be9073e70f064a84b2b0f1854b8db204, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ff75152bef214b51bf249e64e0341f34, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/97d9beddfbca474d9f178711cadcf3bd, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/b18bc41b32274e03a92f7d9f409420b9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6c06424b927e49c88c7077e2a50605ec] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp, totalSize=26.7 K 2023-02-08 23:55:27,876 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting be9073e70f064a84b2b0f1854b8db204, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512659 2023-02-08 23:55:27,876 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting ff75152bef214b51bf249e64e0341f34, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514448 2023-02-08 23:55:27,877 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 97d9beddfbca474d9f178711cadcf3bd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515844 2023-02-08 23:55:27,877 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting b18bc41b32274e03a92f7d9f409420b9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:27,877 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 6c06424b927e49c88c7077e2a50605ec, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 23:55:27,888 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): bf85808cb96a3fd70743466476b776c5#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:27,964 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/4461f76f0e1a4a289ede864a928e7ef2, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7a764657617342fe908e99ce225c7881, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7599740811c241538c4b650f4a4a7446, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/2c3fa0ceb2094c30b0c89a21903c55a6] to archive 2023-02-08 23:55:27,965 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:27,967 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/4461f76f0e1a4a289ede864a928e7ef2 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/4461f76f0e1a4a289ede864a928e7ef2 2023-02-08 23:55:27,968 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7a764657617342fe908e99ce225c7881 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7a764657617342fe908e99ce225c7881 2023-02-08 23:55:27,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7599740811c241538c4b650f4a4a7446 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/7599740811c241538c4b650f4a4a7446 2023-02-08 23:55:27,970 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/5fa5faab97ca4ee48dc3d414eaa6e9bb 2023-02-08 23:55:27,971 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/2c3fa0ceb2094c30b0c89a21903c55a6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/2c3fa0ceb2094c30b0c89a21903c55a6 2023-02-08 23:55:28,297 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302087933e698bbb74222abb2ea51e8471987_del 2023-02-08 23:55:28,297 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302087933e698bbb74222abb2ea51e8471987_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302087933e698bbb74222abb2ea51e8471987_del 2023-02-08 23:55:29,111 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ec47e9094d04598a2ce5a14ff24f762 2023-02-08 23:55:29,111 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/6ec47e9094d04598a2ce5a14ff24f762 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6ec47e9094d04598a2ce5a14ff24f762 2023-02-08 23:55:29,115 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ec47e9094d04598a2ce5a14ff24f762 2023-02-08 23:55:29,116 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bf85808cb96a3fd70743466476b776c5/family1 of bf85808cb96a3fd70743466476b776c5 into 6ec47e9094d04598a2ce5a14ff24f762(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:29,116 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:29,116 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:29,116 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): bf85808cb96a3fd70743466476b776c5/family2 is initiating major compaction (all files) 2023-02-08 23:55:29,116 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:29,116 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:29,116 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of bf85808cb96a3fd70743466476b776c5/family2 in testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:29,117 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/f6702cb56c6049bdb1dcc89a1b85895d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/5e0aadad3d8e4d40a9a856d5d0e9935d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/c6f4612529414554b9acfc9c01140bb5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/20b322639109490b8cafc695fbc09c2f, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/aa9ee19e33a5485c8247adf9007236ea] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp, totalSize=25.4 K 2023-02-08 23:55:29,117 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting f6702cb56c6049bdb1dcc89a1b85895d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512659 2023-02-08 23:55:29,117 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 5e0aadad3d8e4d40a9a856d5d0e9935d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514448 2023-02-08 23:55:29,118 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting c6f4612529414554b9acfc9c01140bb5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515844 2023-02-08 23:55:29,118 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 20b322639109490b8cafc695fbc09c2f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:29,118 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting aa9ee19e33a5485c8247adf9007236ea, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 23:55:29,128 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): bf85808cb96a3fd70743466476b776c5#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:29,155 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/be9073e70f064a84b2b0f1854b8db204, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ff75152bef214b51bf249e64e0341f34, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/97d9beddfbca474d9f178711cadcf3bd, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/b18bc41b32274e03a92f7d9f409420b9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6c06424b927e49c88c7077e2a50605ec] to archive 2023-02-08 23:55:29,156 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:29,158 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/be9073e70f064a84b2b0f1854b8db204 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/be9073e70f064a84b2b0f1854b8db204 2023-02-08 23:55:29,159 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ff75152bef214b51bf249e64e0341f34 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ff75152bef214b51bf249e64e0341f34 2023-02-08 23:55:29,159 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/97d9beddfbca474d9f178711cadcf3bd to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/97d9beddfbca474d9f178711cadcf3bd 2023-02-08 23:55:29,160 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/b18bc41b32274e03a92f7d9f409420b9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/b18bc41b32274e03a92f7d9f409420b9 2023-02-08 23:55:29,161 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6c06424b927e49c88c7077e2a50605ec to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6c06424b927e49c88c7077e2a50605ec 2023-02-08 23:55:29,538 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302086cd700d3848d42268973f64c056407f0_del 2023-02-08 23:55:29,538 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302086cd700d3848d42268973f64c056407f0_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302086cd700d3848d42268973f64c056407f0_del 2023-02-08 23:55:30,353 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bec3267080e8464daf413c218c31dc82 2023-02-08 23:55:30,353 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/bec3267080e8464daf413c218c31dc82 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/bec3267080e8464daf413c218c31dc82 2023-02-08 23:55:30,357 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bec3267080e8464daf413c218c31dc82 2023-02-08 23:55:30,358 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bf85808cb96a3fd70743466476b776c5/family2 of bf85808cb96a3fd70743466476b776c5 into bec3267080e8464daf413c218c31dc82(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:30,358 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:30,358 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:30,358 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): c0432a17915eb080602d94742abd61bf/family1 is initiating major compaction (all files) 2023-02-08 23:55:30,358 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:30,358 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:30,359 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of c0432a17915eb080602d94742abd61bf/family1 in testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:30,359 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0e38670639294b138a863ec328b3514c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/904a84fc1b8442109aa95d92a5281578, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/388d3ec4cc4041828e2fe571247f5aad, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/d29e85ef6e604a50a4aa2d61a274e51f, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/630524251a304b0f836d782eab670beb] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp, totalSize=26.7 K 2023-02-08 23:55:30,359 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 0e38670639294b138a863ec328b3514c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512662 2023-02-08 23:55:30,360 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 904a84fc1b8442109aa95d92a5281578, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514449 2023-02-08 23:55:30,360 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 388d3ec4cc4041828e2fe571247f5aad, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515845 2023-02-08 23:55:30,360 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting d29e85ef6e604a50a4aa2d61a274e51f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:30,361 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 630524251a304b0f836d782eab670beb, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 23:55:30,371 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): c0432a17915eb080602d94742abd61bf#family1#compaction#111 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:30,461 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/f6702cb56c6049bdb1dcc89a1b85895d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/5e0aadad3d8e4d40a9a856d5d0e9935d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/c6f4612529414554b9acfc9c01140bb5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/20b322639109490b8cafc695fbc09c2f, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/aa9ee19e33a5485c8247adf9007236ea] to archive 2023-02-08 23:55:30,462 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:30,463 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/f6702cb56c6049bdb1dcc89a1b85895d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/f6702cb56c6049bdb1dcc89a1b85895d 2023-02-08 23:55:30,464 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/5e0aadad3d8e4d40a9a856d5d0e9935d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/5e0aadad3d8e4d40a9a856d5d0e9935d 2023-02-08 23:55:30,465 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/c6f4612529414554b9acfc9c01140bb5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/c6f4612529414554b9acfc9c01140bb5 2023-02-08 23:55:30,467 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/20b322639109490b8cafc695fbc09c2f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/20b322639109490b8cafc695fbc09c2f 2023-02-08 23:55:30,468 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/aa9ee19e33a5485c8247adf9007236ea to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/aa9ee19e33a5485c8247adf9007236ea 2023-02-08 23:55:30,781 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230208b353ed806a924fa3b0186847a3fdb26a_del 2023-02-08 23:55:30,781 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208b353ed806a924fa3b0186847a3fdb26a_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208b353ed806a924fa3b0186847a3fdb26a_del 2023-02-08 23:55:31,595 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0214826cfc254c82be03999cf8abe3f1 2023-02-08 23:55:31,595 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/0214826cfc254c82be03999cf8abe3f1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0214826cfc254c82be03999cf8abe3f1 2023-02-08 23:55:31,599 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0214826cfc254c82be03999cf8abe3f1 2023-02-08 23:55:31,600 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c0432a17915eb080602d94742abd61bf/family1 of c0432a17915eb080602d94742abd61bf into 0214826cfc254c82be03999cf8abe3f1(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:31,600 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:31,600 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:31,600 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): c0432a17915eb080602d94742abd61bf/family2 is initiating major compaction (all files) 2023-02-08 23:55:31,600 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:31,600 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:31,601 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of c0432a17915eb080602d94742abd61bf/family2 in testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:31,601 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/2d7243117fd7456eaf23062495841f44, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/596a8c7e9ee74ee9a9c70737892c7b88, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/73bfac9ee9854508b7540975ac4e8772, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/0725b9bf527c498bb05cd7b12e41b68a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/aa53a1d627a14499bb9a101f0b9e23ea] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp, totalSize=25.4 K 2023-02-08 23:55:31,601 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 2d7243117fd7456eaf23062495841f44, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512662 2023-02-08 23:55:31,602 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 596a8c7e9ee74ee9a9c70737892c7b88, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514449 2023-02-08 23:55:31,602 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 73bfac9ee9854508b7540975ac4e8772, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515845 2023-02-08 23:55:31,602 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 0725b9bf527c498bb05cd7b12e41b68a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:31,603 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting aa53a1d627a14499bb9a101f0b9e23ea, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 23:55:31,615 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): c0432a17915eb080602d94742abd61bf#family2#compaction#112 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:31,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0e38670639294b138a863ec328b3514c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/904a84fc1b8442109aa95d92a5281578, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/388d3ec4cc4041828e2fe571247f5aad, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/d29e85ef6e604a50a4aa2d61a274e51f, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/630524251a304b0f836d782eab670beb] to archive 2023-02-08 23:55:31,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:31,654 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0e38670639294b138a863ec328b3514c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0e38670639294b138a863ec328b3514c 2023-02-08 23:55:31,655 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/904a84fc1b8442109aa95d92a5281578 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/904a84fc1b8442109aa95d92a5281578 2023-02-08 23:55:31,655 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/388d3ec4cc4041828e2fe571247f5aad to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/388d3ec4cc4041828e2fe571247f5aad 2023-02-08 23:55:31,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/d29e85ef6e604a50a4aa2d61a274e51f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/d29e85ef6e604a50a4aa2d61a274e51f 2023-02-08 23:55:31,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/630524251a304b0f836d782eab670beb to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/630524251a304b0f836d782eab670beb 2023-02-08 23:55:32,024 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020892d24b79283a4e7a9a20f26c21823788_del 2023-02-08 23:55:32,025 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020892d24b79283a4e7a9a20f26c21823788_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020892d24b79283a4e7a9a20f26c21823788_del 2023-02-08 23:55:32,442 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78ac432ab5764403939baea9569f1f25 2023-02-08 23:55:32,443 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/78ac432ab5764403939baea9569f1f25 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/78ac432ab5764403939baea9569f1f25 2023-02-08 23:55:32,446 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78ac432ab5764403939baea9569f1f25 2023-02-08 23:55:32,447 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c0432a17915eb080602d94742abd61bf/family2 of c0432a17915eb080602d94742abd61bf into 78ac432ab5764403939baea9569f1f25(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:32,447 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:32,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/2d7243117fd7456eaf23062495841f44, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/596a8c7e9ee74ee9a9c70737892c7b88, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/73bfac9ee9854508b7540975ac4e8772, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/0725b9bf527c498bb05cd7b12e41b68a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/aa53a1d627a14499bb9a101f0b9e23ea] to archive 2023-02-08 23:55:32,452 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:32,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:32,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/2d7243117fd7456eaf23062495841f44 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/2d7243117fd7456eaf23062495841f44 2023-02-08 23:55:32,455 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:32,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/596a8c7e9ee74ee9a9c70737892c7b88 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/596a8c7e9ee74ee9a9c70737892c7b88 2023-02-08 23:55:32,458 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/73bfac9ee9854508b7540975ac4e8772 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/73bfac9ee9854508b7540975ac4e8772 2023-02-08 23:55:32,459 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/0725b9bf527c498bb05cd7b12e41b68a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/0725b9bf527c498bb05cd7b12e41b68a 2023-02-08 23:55:32,460 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/aa53a1d627a14499bb9a101f0b9e23ea to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/aa53a1d627a14499bb9a101f0b9e23ea 2023-02-08 23:55:32,460 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:32,461 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:32,469 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 23:55:32,504 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-08 23:55:32,507 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302087933e698bbb74222abb2ea51e8471987_del 2023-02-08 23:55:32,510 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302086fdac96439a746ee9630abc2ec957784_del 2023-02-08 23:55:32,513 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230208b353ed806a924fa3b0186847a3fdb26a_del 2023-02-08 23:55:32,513 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:32,514 DEBUG [Listener at localhost/41521] zookeeper.ReadOnlyZKClient(139): Connect 0x78e7e520 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:55:32,519 DEBUG [Listener at localhost/41521] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f1d5058, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:55:32,519 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:32,519 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:32,519 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:33,376 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020839c46593e81b40bfbdff20036bd440e0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020839c46593e81b40bfbdff20036bd440e0 2023-02-08 23:55:33,376 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208a25077352895405b87fd1c55205e32a5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208a25077352895405b87fd1c55205e32a5 2023-02-08 23:55:33,377 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208c396c80f3aba4ac5a0065c4c3651d0c7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208c396c80f3aba4ac5a0065c4c3651d0c7 2023-02-08 23:55:33,377 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:33,377 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:33,377 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:33,392 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:55:33,396 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53128, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:55:33,401 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,401 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,401 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,401 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,402 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:55:33,404 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59698, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:55:33,404 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,404 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:33,405 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,405 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:33,415 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/7f47b43816cc4a7b8b5b48015167f560 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-08 23:55:33,415 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/9478d2fad0d14e6784accfbe44cebb07 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-08 23:55:33,415 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/0850fcff42184daeafcbbc54cc0f38a1 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-08 23:55:33,417 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/7f47b43816cc4a7b8b5b48015167f560}] 2023-02-08 23:55:33,417 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/9478d2fad0d14e6784accfbe44cebb07}] 2023-02-08 23:55:33,417 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/0850fcff42184daeafcbbc54cc0f38a1}] 2023-02-08 23:55:33,423 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/9478d2fad0d14e6784accfbe44cebb07 for inclusion in c0432a17915eb080602d94742abd61bf/family1 2023-02-08 23:55:33,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/0850fcff42184daeafcbbc54cc0f38a1 for inclusion in 83e0134f0861e917b7a0acc88b520e50/family1 2023-02-08 23:55:33,423 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/7f47b43816cc4a7b8b5b48015167f560 for inclusion in bf85808cb96a3fd70743466476b776c5/family1 2023-02-08 23:55:33,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-08 23:55:33,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 23:55:33,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-08 23:55:33,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-08 23:55:33,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last=1 2023-02-08 23:55:33,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-08 23:55:33,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:33,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:33,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:33,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/7f47b43816cc4a7b8b5b48015167f560 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__mbsj7kq786uuj0g4a95q34vsk72ek5bp38r6k5u8qfm02jnmd9vot79re2adsraj/family1/7f47b43816cc4a7b8b5b48015167f560 2023-02-08 23:55:33,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/9478d2fad0d14e6784accfbe44cebb07 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__iqn9sf3fh629jl93dbgk5j0170vclasafe3d2h9vedaekclpldrno1jhk4fkj1b6/family1/9478d2fad0d14e6784accfbe44cebb07 2023-02-08 23:55:33,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/0850fcff42184daeafcbbc54cc0f38a1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7mi52inbcl3ksiav9m55pdppf25gtqj79b7ck26suug3bo4sjeo3mmj5f83hoifd/family1/0850fcff42184daeafcbbc54cc0f38a1 2023-02-08 23:55:33,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__mbsj7kq786uuj0g4a95q34vsk72ek5bp38r6k5u8qfm02jnmd9vot79re2adsraj/family1/7f47b43816cc4a7b8b5b48015167f560 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ab890ed0ab7f4848ac4497a2c5078b8e_SeqId_29_ 2023-02-08 23:55:33,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__iqn9sf3fh629jl93dbgk5j0170vclasafe3d2h9vedaekclpldrno1jhk4fkj1b6/family1/9478d2fad0d14e6784accfbe44cebb07 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/1e36366b21064206b0af81bdb13c3a8c_SeqId_29_ 2023-02-08 23:55:33,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7mi52inbcl3ksiav9m55pdppf25gtqj79b7ck26suug3bo4sjeo3mmj5f83hoifd/family1/0850fcff42184daeafcbbc54cc0f38a1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/79723100e51f4789b21dc3f6edab7ae6_SeqId_29_ 2023-02-08 23:55:33,430 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__iqn9sf3fh629jl93dbgk5j0170vclasafe3d2h9vedaekclpldrno1jhk4fkj1b6/family1/9478d2fad0d14e6784accfbe44cebb07 into c0432a17915eb080602d94742abd61bf/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/1e36366b21064206b0af81bdb13c3a8c_SeqId_29_ - updating store file list. 2023-02-08 23:55:33,431 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__mbsj7kq786uuj0g4a95q34vsk72ek5bp38r6k5u8qfm02jnmd9vot79re2adsraj/family1/7f47b43816cc4a7b8b5b48015167f560 into bf85808cb96a3fd70743466476b776c5/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ab890ed0ab7f4848ac4497a2c5078b8e_SeqId_29_ - updating store file list. 2023-02-08 23:55:33,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7mi52inbcl3ksiav9m55pdppf25gtqj79b7ck26suug3bo4sjeo3mmj5f83hoifd/family1/0850fcff42184daeafcbbc54cc0f38a1 into 83e0134f0861e917b7a0acc88b520e50/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/79723100e51f4789b21dc3f6edab7ae6_SeqId_29_ - updating store file list. 2023-02-08 23:55:33,434 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/1e36366b21064206b0af81bdb13c3a8c_SeqId_29_ into c0432a17915eb080602d94742abd61bf/family1 2023-02-08 23:55:33,434 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__iqn9sf3fh629jl93dbgk5j0170vclasafe3d2h9vedaekclpldrno1jhk4fkj1b6/family1/9478d2fad0d14e6784accfbe44cebb07 into c0432a17915eb080602d94742abd61bf/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/1e36366b21064206b0af81bdb13c3a8c_SeqId_29_) 2023-02-08 23:55:33,434 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ab890ed0ab7f4848ac4497a2c5078b8e_SeqId_29_ into bf85808cb96a3fd70743466476b776c5/family1 2023-02-08 23:55:33,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/79723100e51f4789b21dc3f6edab7ae6_SeqId_29_ into 83e0134f0861e917b7a0acc88b520e50/family1 2023-02-08 23:55:33,434 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__mbsj7kq786uuj0g4a95q34vsk72ek5bp38r6k5u8qfm02jnmd9vot79re2adsraj/family1/7f47b43816cc4a7b8b5b48015167f560 into bf85808cb96a3fd70743466476b776c5/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ab890ed0ab7f4848ac4497a2c5078b8e_SeqId_29_) 2023-02-08 23:55:33,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7mi52inbcl3ksiav9m55pdppf25gtqj79b7ck26suug3bo4sjeo3mmj5f83hoifd/family1/0850fcff42184daeafcbbc54cc0f38a1 into 83e0134f0861e917b7a0acc88b520e50/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/79723100e51f4789b21dc3f6edab7ae6_SeqId_29_) 2023-02-08 23:55:33,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__iqn9sf3fh629jl93dbgk5j0170vclasafe3d2h9vedaekclpldrno1jhk4fkj1b6/family1/9478d2fad0d14e6784accfbe44cebb07 2023-02-08 23:55:33,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__mbsj7kq786uuj0g4a95q34vsk72ek5bp38r6k5u8qfm02jnmd9vot79re2adsraj/family1/7f47b43816cc4a7b8b5b48015167f560 2023-02-08 23:55:33,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7mi52inbcl3ksiav9m55pdppf25gtqj79b7ck26suug3bo4sjeo3mmj5f83hoifd/family1/0850fcff42184daeafcbbc54cc0f38a1 2023-02-08 23:55:33,436 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:33,437 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:33,437 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:33,437 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:33,437 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:33,438 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:33,439 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302080854ba64b08e4a44a8ef84237326c925 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302080854ba64b08e4a44a8ef84237326c925 2023-02-08 23:55:33,439 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020845886db6ce904d17bca6173c2e44fa1c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023020845886db6ce904d17bca6173c2e44fa1c 2023-02-08 23:55:33,440 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302084c94eb80abf24e1f920601f01f56cda5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302084c94eb80abf24e1f920601f01f56cda5 2023-02-08 23:55:33,440 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020824290db562504ae099dc7ac42a91428a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020824290db562504ae099dc7ac42a91428a 2023-02-08 23:55:33,440 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088eb3af42556a4def9b2293f1392ac0a3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088eb3af42556a4def9b2293f1392ac0a3 2023-02-08 23:55:33,442 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302086879d9029967492f881893b3ece54a05 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302086879d9029967492f881893b3ece54a05 2023-02-08 23:55:33,442 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302082fa734f47d4b4337ace2dfaa02fb8c4a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302082fa734f47d4b4337ace2dfaa02fb8c4a 2023-02-08 23:55:33,442 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088edfd6d65d9c4a9c9ada8e78ee01fb64 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302088edfd6d65d9c4a9c9ada8e78ee01fb64 2023-02-08 23:55:33,443 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020896ce51894cc442cbaafeb262785520c6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020896ce51894cc442cbaafeb262785520c6 2023-02-08 23:55:33,443 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302084298f8536b12414885004ca6363d5751 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302084298f8536b12414885004ca6363d5751 2023-02-08 23:55:33,444 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 23:55:33,444 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 23:55:33,444 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208b67eff96efab410aa112021b51ca7dfc to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208b67eff96efab410aa112021b51ca7dfc 2023-02-08 23:55:33,444 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 23:55:33,444 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 23:55:33,444 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:33,444 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:33,445 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208eee36449625442d0980dcaeb5c877f3f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208eee36449625442d0980dcaeb5c877f3f 2023-02-08 23:55:33,445 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 23:55:33,445 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208' 2023-02-08 23:55:33,445 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:33,445 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:33,445 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:33,445 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(367): [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302087933e698bbb74222abb2ea51e8471987_del] 2023-02-08 23:55:33,446 DEBUG [Listener at localhost/41521] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:33,447 DEBUG [Listener at localhost/41521] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302087933e698bbb74222abb2ea51e8471987_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302087933e698bbb74222abb2ea51e8471987_del 2023-02-08 23:55:33,447 INFO [Listener at localhost/41521] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-08 23:55:33,447 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(367): [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302086fdac96439a746ee9630abc2ec957784_del] 2023-02-08 23:55:33,448 DEBUG [Listener at localhost/41521] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:33,449 DEBUG [Listener at localhost/41521] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302086fdac96439a746ee9630abc2ec957784_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302086fdac96439a746ee9630abc2ec957784_del 2023-02-08 23:55:33,449 INFO [Listener at localhost/41521] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-08 23:55:33,449 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(367): [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208b353ed806a924fa3b0186847a3fdb26a_del] 2023-02-08 23:55:33,449 DEBUG [Listener at localhost/41521] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:33,450 DEBUG [Listener at localhost/41521] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208b353ed806a924fa3b0186847a3fdb26a_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208b353ed806a924fa3b0186847a3fdb26a_del 2023-02-08 23:55:33,450 INFO [Listener at localhost/41521] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-08 23:55:33,454 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:33,456 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:33,460 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:33,461 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:33,468 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,468 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:33,468 INFO [Listener at localhost/41521] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-08 23:55:33,469 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-02-08 23:55:33,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-08 23:55:33,471 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,471 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:33,472 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900533472"}]},"ts":"1675900533472"} 2023-02-08 23:55:33,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 23:55:33,473 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-08 23:55:33,475 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-08 23:55:33,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, UNASSIGN}] 2023-02-08 23:55:33,477 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, UNASSIGN 2023-02-08 23:55:33,477 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, UNASSIGN 2023-02-08 23:55:33,477 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, UNASSIGN 2023-02-08 23:55:33,478 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=83e0134f0861e917b7a0acc88b520e50, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:33,478 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=bf85808cb96a3fd70743466476b776c5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:33,478 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=c0432a17915eb080602d94742abd61bf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:33,478 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900533478"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900533478"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900533478"}]},"ts":"1675900533478"} 2023-02-08 23:55:33,478 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900533478"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900533478"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900533478"}]},"ts":"1675900533478"} 2023-02-08 23:55:33,478 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900533478"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900533478"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900533478"}]},"ts":"1675900533478"} 2023-02-08 23:55:33,479 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure 83e0134f0861e917b7a0acc88b520e50, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:33,480 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure bf85808cb96a3fd70743466476b776c5, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:33,480 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure c0432a17915eb080602d94742abd61bf, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:33,572 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,572 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:33,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 23:55:33,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:33,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c0432a17915eb080602d94742abd61bf, disabling compactions & flushes 2023-02-08 23:55:33,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:33,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:33,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. after waiting 0 ms 2023-02-08 23:55:33,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:33,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-08 23:55:33,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:33,639 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c0432a17915eb080602d94742abd61bf: Waiting for close lock at 1675900533631Running coprocessor pre-close hooks at 1675900533631Disabling compacts and flushes for region at 1675900533631Disabling writes for close at 1675900533631Writing region close event to WAL at 1675900533635 (+4 ms)Running coprocessor post-close hooks at 1675900533639 (+4 ms)Closed at 1675900533639 2023-02-08 23:55:33,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:33,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:33,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 83e0134f0861e917b7a0acc88b520e50, disabling compactions & flushes 2023-02-08 23:55:33,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:33,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:33,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. after waiting 0 ms 2023-02-08 23:55:33,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:33,641 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=c0432a17915eb080602d94742abd61bf, regionState=CLOSED 2023-02-08 23:55:33,641 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900533641"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900533641"}]},"ts":"1675900533641"} 2023-02-08 23:55:33,645 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-02-08 23:55:33,645 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure c0432a17915eb080602d94742abd61bf, server=jenkins-hbase4.apache.org,42703,1675900414768 in 163 msec 2023-02-08 23:55:33,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-08 23:55:33,646 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, UNASSIGN in 169 msec 2023-02-08 23:55:33,646 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:33,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 83e0134f0861e917b7a0acc88b520e50: Waiting for close lock at 1675900533641Running coprocessor pre-close hooks at 1675900533641Disabling compacts and flushes for region at 1675900533641Disabling writes for close at 1675900533641Writing region close event to WAL at 1675900533643 (+2 ms)Running coprocessor post-close hooks at 1675900533646 (+3 ms)Closed at 1675900533646 2023-02-08 23:55:33,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:33,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:33,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bf85808cb96a3fd70743466476b776c5, disabling compactions & flushes 2023-02-08 23:55:33,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:33,647 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=83e0134f0861e917b7a0acc88b520e50, regionState=CLOSED 2023-02-08 23:55:33,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:33,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. after waiting 0 ms 2023-02-08 23:55:33,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:33,648 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900533647"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900533647"}]},"ts":"1675900533647"} 2023-02-08 23:55:33,651 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-08 23:55:33,651 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure 83e0134f0861e917b7a0acc88b520e50, server=jenkins-hbase4.apache.org,42703,1675900414768 in 171 msec 2023-02-08 23:55:33,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, UNASSIGN in 175 msec 2023-02-08 23:55:33,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-08 23:55:33,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:33,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bf85808cb96a3fd70743466476b776c5: Waiting for close lock at 1675900533647Running coprocessor pre-close hooks at 1675900533647Disabling compacts and flushes for region at 1675900533647Disabling writes for close at 1675900533648 (+1 ms)Writing region close event to WAL at 1675900533650 (+2 ms)Running coprocessor post-close hooks at 1675900533652 (+2 ms)Closed at 1675900533653 (+1 ms) 2023-02-08 23:55:33,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:33,654 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=bf85808cb96a3fd70743466476b776c5, regionState=CLOSED 2023-02-08 23:55:33,654 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900533654"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900533654"}]},"ts":"1675900533654"} 2023-02-08 23:55:33,656 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=89 2023-02-08 23:55:33,656 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure bf85808cb96a3fd70743466476b776c5, server=jenkins-hbase4.apache.org,42703,1675900414768 in 175 msec 2023-02-08 23:55:33,657 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-08 23:55:33,657 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, UNASSIGN in 180 msec 2023-02-08 23:55:33,658 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900533658"}]},"ts":"1675900533658"} 2023-02-08 23:55:33,659 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-08 23:55:33,660 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-08 23:55:33,661 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 191 msec 2023-02-08 23:55:33,774 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,774 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:33,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 23:55:33,775 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-08 23:55:33,776 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,776 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:33,923 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-08 23:55:33,923 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:55:33,923 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x78e7e520 to 127.0.0.1:54774 2023-02-08 23:55:33,924 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:33,924 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:55:33,924 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1fef556f to 127.0.0.1:54774 2023-02-08 23:55:33,924 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:33,924 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x67693ffb to 127.0.0.1:54774 2023-02-08 23:55:33,924 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:33,924 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:55:33,924 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4278bf06 to 127.0.0.1:54774 2023-02-08 23:55:33,924 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:33,925 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:55:33,925 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5b06711e to 127.0.0.1:54774 2023-02-08 23:55:33,925 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:33,932 INFO [Listener at localhost/41521] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1675900519425-1675900533932 2023-02-08 23:55:33,932 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:33,932 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:33,933 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 23:55:33,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675900533933 (current time:1675900533933). 2023-02-08 23:55:33,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-08 23:55:33,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] zookeeper.ReadOnlyZKClient(139): Connect 0x22843944 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:55:33,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c143795, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:55:33,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:55:33,941 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53138, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:55:33,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x22843944 to 127.0.0.1:54774 2023-02-08 23:55:33,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:55:33,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-08 23:55:33,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 23:55:33,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 23:55:33,945 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 23:55:33,945 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675901133945 2023-02-08 23:55:33,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 23:55:33,945 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1675900519425-1675900533932 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-08 23:55:33,946 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:33,946 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-08 23:55:34,046 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:34,046 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,046 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:34,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 23:55:34,047 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-08 23:55:34,247 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:34,247 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,247 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:34,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 23:55:34,249 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-08 23:55:34,352 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-08 23:55:34,354 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-08 23:55:34,354 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-08 23:55:34,354 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-08 23:55:34,354 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-08 23:55:34,354 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-08 23:55:34,354 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-08 23:55:34,354 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-08 23:55:34,354 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0214826cfc254c82be03999cf8abe3f1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/1e36366b21064206b0af81bdb13c3a8c_SeqId_29_] hfiles 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6ec47e9094d04598a2ce5a14ff24f762, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ab890ed0ab7f4848ac4497a2c5078b8e_SeqId_29_] hfiles 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0214826cfc254c82be03999cf8abe3f1 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6ec47e9094d04598a2ce5a14ff24f762 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/1e36366b21064206b0af81bdb13c3a8c_SeqId_29_ 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ab890ed0ab7f4848ac4497a2c5078b8e_SeqId_29_ 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/69c7c8ca5ef14025be53d442401acd49, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/79723100e51f4789b21dc3f6edab7ae6_SeqId_29_] hfiles 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/69c7c8ca5ef14025be53d442401acd49 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/79723100e51f4789b21dc3f6edab7ae6_SeqId_29_ 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208c396c80f3aba4ac5a0065c4c3651d0c7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020839c46593e81b40bfbdff20036bd440e0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208a25077352895405b87fd1c55205e32a5] hfiles 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208c396c80f3aba4ac5a0065c4c3651d0c7 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020839c46593e81b40bfbdff20036bd440e0 2023-02-08 23:55:34,357 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208a25077352895405b87fd1c55205e32a5 2023-02-08 23:55:34,358 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/78ac432ab5764403939baea9569f1f25] hfiles 2023-02-08 23:55:34,358 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/bec3267080e8464daf413c218c31dc82] hfiles 2023-02-08 23:55:34,358 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/78ac432ab5764403939baea9569f1f25 2023-02-08 23:55:34,358 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/bec3267080e8464daf413c218c31dc82 2023-02-08 23:55:34,358 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/15d177a2fbf0481587bafc06b15c6690] hfiles 2023-02-08 23:55:34,358 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/15d177a2fbf0481587bafc06b15c6690 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302083870c496f39e4adead682c1808ae9f03, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020863c747205e36467b845e75e62af56bbc, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020893dba873f1e74ce5afc7e102e2fbf73d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208c6b4d5234e7a421098a7c9f75c24ab74_del, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208fb345cb5d4514f6284d52b663db4ff57, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020826e765438cb541d8b814ee8f7e8168ac, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020892d24b79283a4e7a9a20f26c21823788_del, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302089eda5bf68c95405c952b826fa83c8e5a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208e2df64581a9744a19609b126756323be, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208eeeb89abf9c5490299cfb4b07b224efd, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208539c497a44e34437b0679f61feaab580, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302086cd700d3848d42268973f64c056407f0_del, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208c4561f861fb945cc896cdeb3f74c35e1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfa91b29aad549c0ad57bb2dc1be2404, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfacfd5ce0334e21ac2a185385ca0c86] hfiles 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302083870c496f39e4adead682c1808ae9f03 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020863c747205e36467b845e75e62af56bbc 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020893dba873f1e74ce5afc7e102e2fbf73d 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208c6b4d5234e7a421098a7c9f75c24ab74_del 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208fb345cb5d4514f6284d52b663db4ff57 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020826e765438cb541d8b814ee8f7e8168ac 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020892d24b79283a4e7a9a20f26c21823788_del 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302089eda5bf68c95405c952b826fa83c8e5a 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208e2df64581a9744a19609b126756323be 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230208eeeb89abf9c5490299cfb4b07b224efd 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208539c497a44e34437b0679f61feaab580 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302086cd700d3848d42268973f64c056407f0_del 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208c4561f861fb945cc896cdeb3f74c35e1 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfa91b29aad549c0ad57bb2dc1be2404 2023-02-08 23:55:34,365 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208cfacfd5ce0334e21ac2a185385ca0c86 2023-02-08 23:55:34,549 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:34,549 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:34,549 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:34,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:34,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 23:55:34,550 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-08 23:55:34,769 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-08 23:55:34,769 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1675900519425-1675900533932 2023-02-08 23:55:34,769 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675900519425-1675900533932 2023-02-08 23:55:35,050 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:35,050 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:35,051 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:35,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:35,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 23:55:35,051 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-08 23:55:35,205 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675900519425-1675900533932 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/hbase-failsafe-snaptb-1675900519425-1675900533932 2023-02-08 23:55:35,944 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 23:55:36,040 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1675900519425-1675900533932 of table testCompactionWithHFileLink completed 2023-02-08 23:55:36,040 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675900519425-1675900533932 2023-02-08 23:55:36,041 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675900519425-1675900533932 2023-02-08 23:55:36,041 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1675900519425-1675900533932 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675900533945Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1675900534352 (+407 ms)Consolidate snapshot: hbase-failsafe-snaptb-1675900519425-1675900533932 at 1675900534769 (+417 ms)Loading Region manifests for hbase-failsafe-snaptb-1675900519425-1675900533932 at 1675900534769Writing data manifest for hbase-failsafe-snaptb-1675900519425-1675900533932 at 1675900534772 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1675900519425-1675900533932 at 1675900535178 (+406 ms)Snapshot hbase-failsafe-snaptb-1675900519425-1675900533932 of table testCompactionWithHFileLink completed at 1675900536040 (+862 ms) 2023-02-08 23:55:36,042 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 23:55:36,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0990 sec 2023-02-08 23:55:36,052 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 23:55:36,052 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,052 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 23:55:36,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1675900519425-1675900533932 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-08 23:55:36,053 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,053 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675900519425" table: "testCompactionWithHFileLink" creation_time: 1675900519435 type: FLUSH version: 2 ttl: 0 ) 2023-02-08 23:55:36,109 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1675900519425 as table=testCompactionWithHFileLink 2023-02-08 23:55:36,111 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,111 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 23:55:36,212 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,213 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 23:55:36,414 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,414 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 23:55:36,519 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-08 23:55:36,520 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:36,520 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-08 23:55:36,521 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 23:55:36,527 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1675900519425" table: "testCompactionWithHFileLink" creation_time: 1675900519435 type: FLUSH version: 2 ttl: 0 2023-02-08 23:55:36,527 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink 2023-02-08 23:55:36,533 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-08 23:55:36,536 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208c396c80f3aba4ac5a0065c4c3651d0c7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208c396c80f3aba4ac5a0065c4c3651d0c7 2023-02-08 23:55:36,538 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208a25077352895405b87fd1c55205e32a5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208a25077352895405b87fd1c55205e32a5 2023-02-08 23:55:36,540 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020839c46593e81b40bfbdff20036bd440e0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020839c46593e81b40bfbdff20036bd440e0 2023-02-08 23:55:36,540 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302084c94eb80abf24e1f920601f01f56cda5 of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,544 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302086879d9029967492f881893b3ece54a05 of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,548 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023020896ce51894cc442cbaafeb262785520c6 of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,552 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230208eee36449625442d0980dcaeb5c877f3f of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,555 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302080854ba64b08e4a44a8ef84237326c925 of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,559 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023020824290db562504ae099dc7ac42a91428a of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,562 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302082fa734f47d4b4337ace2dfaa02fb8c4a of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,565 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302084298f8536b12414885004ca6363d5751 of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,568 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023020845886db6ce904d17bca6173c2e44fa1c of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,570 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302088eb3af42556a4def9b2293f1392ac0a3 of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,573 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302088edfd6d65d9c4a9c9ada8e78ee01fb64 of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,576 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230208b67eff96efab410aa112021b51ca7dfc of snapshot=snaptb-1675900519425 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 23:55:36,581 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020892d24b79283a4e7a9a20f26c21823788_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020892d24b79283a4e7a9a20f26c21823788_del 2023-02-08 23:55:36,582 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208c6b4d5234e7a421098a7c9f75c24ab74_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208c6b4d5234e7a421098a7c9f75c24ab74_del 2023-02-08 23:55:36,583 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302086cd700d3848d42268973f64c056407f0_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302086cd700d3848d42268973f64c056407f0_del 2023-02-08 23:55:36,583 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:36,583 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:36,583 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:36,586 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/69c7c8ca5ef14025be53d442401acd49 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/69c7c8ca5ef14025be53d442401acd49 2023-02-08 23:55:36,586 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6ec47e9094d04598a2ce5a14ff24f762 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/6ec47e9094d04598a2ce5a14ff24f762 2023-02-08 23:55:36,586 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0214826cfc254c82be03999cf8abe3f1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/0214826cfc254c82be03999cf8abe3f1 2023-02-08 23:55:36,588 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/79723100e51f4789b21dc3f6edab7ae6_SeqId_29_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/79723100e51f4789b21dc3f6edab7ae6_SeqId_29_ 2023-02-08 23:55:36,588 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink efc80c1c81864c3990e1fec54dc689c0 of snapshot=snaptb-1675900519425 to region=83e0134f0861e917b7a0acc88b520e50 table=testCompactionWithHFileLink 2023-02-08 23:55:36,588 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ab890ed0ab7f4848ac4497a2c5078b8e_SeqId_29_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ab890ed0ab7f4848ac4497a2c5078b8e_SeqId_29_ 2023-02-08 23:55:36,588 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink be9073e70f064a84b2b0f1854b8db204 of snapshot=snaptb-1675900519425 to region=bf85808cb96a3fd70743466476b776c5 table=testCompactionWithHFileLink 2023-02-08 23:55:36,588 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/1e36366b21064206b0af81bdb13c3a8c_SeqId_29_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/1e36366b21064206b0af81bdb13c3a8c_SeqId_29_ 2023-02-08 23:55:36,588 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0e38670639294b138a863ec328b3514c of snapshot=snaptb-1675900519425 to region=c0432a17915eb080602d94742abd61bf table=testCompactionWithHFileLink 2023-02-08 23:55:36,594 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ff75152bef214b51bf249e64e0341f34 of snapshot=snaptb-1675900519425 to region=bf85808cb96a3fd70743466476b776c5 table=testCompactionWithHFileLink 2023-02-08 23:55:36,595 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8e816cbaac864e4bac1ad5c1403b2cda of snapshot=snaptb-1675900519425 to region=83e0134f0861e917b7a0acc88b520e50 table=testCompactionWithHFileLink 2023-02-08 23:55:36,596 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 904a84fc1b8442109aa95d92a5281578 of snapshot=snaptb-1675900519425 to region=c0432a17915eb080602d94742abd61bf table=testCompactionWithHFileLink 2023-02-08 23:55:36,598 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 97d9beddfbca474d9f178711cadcf3bd of snapshot=snaptb-1675900519425 to region=bf85808cb96a3fd70743466476b776c5 table=testCompactionWithHFileLink 2023-02-08 23:55:36,599 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 41e307b1667b4e89bec738267d5e3eb7 of snapshot=snaptb-1675900519425 to region=83e0134f0861e917b7a0acc88b520e50 table=testCompactionWithHFileLink 2023-02-08 23:55:36,599 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 388d3ec4cc4041828e2fe571247f5aad of snapshot=snaptb-1675900519425 to region=c0432a17915eb080602d94742abd61bf table=testCompactionWithHFileLink 2023-02-08 23:55:36,601 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b18bc41b32274e03a92f7d9f409420b9 of snapshot=snaptb-1675900519425 to region=bf85808cb96a3fd70743466476b776c5 table=testCompactionWithHFileLink 2023-02-08 23:55:36,602 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9589f8bc35fd4c6e98792433f593e54c of snapshot=snaptb-1675900519425 to region=83e0134f0861e917b7a0acc88b520e50 table=testCompactionWithHFileLink 2023-02-08 23:55:36,602 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d29e85ef6e604a50a4aa2d61a274e51f of snapshot=snaptb-1675900519425 to region=c0432a17915eb080602d94742abd61bf table=testCompactionWithHFileLink 2023-02-08 23:55:36,605 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/bec3267080e8464daf413c218c31dc82 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/bec3267080e8464daf413c218c31dc82 2023-02-08 23:55:36,605 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f6702cb56c6049bdb1dcc89a1b85895d of snapshot=snaptb-1675900519425 to region=bf85808cb96a3fd70743466476b776c5 table=testCompactionWithHFileLink 2023-02-08 23:55:36,606 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/15d177a2fbf0481587bafc06b15c6690 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/15d177a2fbf0481587bafc06b15c6690 2023-02-08 23:55:36,606 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4461f76f0e1a4a289ede864a928e7ef2 of snapshot=snaptb-1675900519425 to region=83e0134f0861e917b7a0acc88b520e50 table=testCompactionWithHFileLink 2023-02-08 23:55:36,607 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/78ac432ab5764403939baea9569f1f25 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/78ac432ab5764403939baea9569f1f25 2023-02-08 23:55:36,607 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2d7243117fd7456eaf23062495841f44 of snapshot=snaptb-1675900519425 to region=c0432a17915eb080602d94742abd61bf table=testCompactionWithHFileLink 2023-02-08 23:55:36,609 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5e0aadad3d8e4d40a9a856d5d0e9935d of snapshot=snaptb-1675900519425 to region=bf85808cb96a3fd70743466476b776c5 table=testCompactionWithHFileLink 2023-02-08 23:55:36,609 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7a764657617342fe908e99ce225c7881 of snapshot=snaptb-1675900519425 to region=83e0134f0861e917b7a0acc88b520e50 table=testCompactionWithHFileLink 2023-02-08 23:55:36,610 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 596a8c7e9ee74ee9a9c70737892c7b88 of snapshot=snaptb-1675900519425 to region=c0432a17915eb080602d94742abd61bf table=testCompactionWithHFileLink 2023-02-08 23:55:36,612 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c6f4612529414554b9acfc9c01140bb5 of snapshot=snaptb-1675900519425 to region=bf85808cb96a3fd70743466476b776c5 table=testCompactionWithHFileLink 2023-02-08 23:55:36,613 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7599740811c241538c4b650f4a4a7446 of snapshot=snaptb-1675900519425 to region=83e0134f0861e917b7a0acc88b520e50 table=testCompactionWithHFileLink 2023-02-08 23:55:36,613 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 73bfac9ee9854508b7540975ac4e8772 of snapshot=snaptb-1675900519425 to region=c0432a17915eb080602d94742abd61bf table=testCompactionWithHFileLink 2023-02-08 23:55:36,615 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 20b322639109490b8cafc695fbc09c2f of snapshot=snaptb-1675900519425 to region=bf85808cb96a3fd70743466476b776c5 table=testCompactionWithHFileLink 2023-02-08 23:55:36,616 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5fa5faab97ca4ee48dc3d414eaa6e9bb of snapshot=snaptb-1675900519425 to region=83e0134f0861e917b7a0acc88b520e50 table=testCompactionWithHFileLink 2023-02-08 23:55:36,616 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0725b9bf527c498bb05cd7b12e41b68a of snapshot=snaptb-1675900519425 to region=c0432a17915eb080602d94742abd61bf table=testCompactionWithHFileLink 2023-02-08 23:55:36,619 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1675900519425" table: "testCompactionWithHFileLink" creation_time: 1675900519435 type: FLUSH version: 2 ttl: 0 2023-02-08 23:55:36,620 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900536620"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:36,621 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900536620"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:36,621 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675900536620"}]},"ts":"9223372036854775807"} 2023-02-08 23:55:36,622 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 23:55:36,622 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 83e0134f0861e917b7a0acc88b520e50, NAME => 'testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => bf85808cb96a3fd70743466476b776c5, NAME => 'testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => c0432a17915eb080602d94742abd61bf, NAME => 'testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.', STARTKEY => '2', ENDKEY => ''}] 2023-02-08 23:55:36,623 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900536621"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900536621"}]},"ts":"1675900536621"} 2023-02-08 23:55:36,623 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900536621"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900536621"}]},"ts":"1675900536621"} 2023-02-08 23:55:36,623 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900536621"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900536621"}]},"ts":"1675900536621"} 2023-02-08 23:55:36,624 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 23:55:36,624 INFO [PEWorker-4] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-08 23:55:36,624 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 83e0134f0861e917b7a0acc88b520e50, NAME => 'testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => bf85808cb96a3fd70743466476b776c5, NAME => 'testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => c0432a17915eb080602d94742abd61bf, NAME => 'testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.', STARTKEY => '2', ENDKEY => ''}] 2023-02-08 23:55:36,625 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1675900519425 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-08 23:55:36,626 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675900519425" table: "testCompactionWithHFileLink" creation_time: 1675900519435 type: FLUSH version: 2 ttl: 0 ) in 525 msec 2023-02-08 23:55:36,715 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,715 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 23:55:36,716 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-08 23:55:36,716 INFO [Listener at localhost/41521] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1675900519425-1675900533932 2023-02-08 23:55:36,717 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,717 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,721 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1675900519425-1675900533932" 2023-02-08 23:55:36,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1675900519425-1675900533932 2023-02-08 23:55:36,725 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,725 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,726 INFO [Listener at localhost/41521] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-08 23:55:36,729 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.HMaster$10(2372): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-02-08 23:55:36,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-08 23:55:36,737 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,737 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,737 INFO [PEWorker-3] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-08 23:55:36,737 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900536737"}]},"ts":"1675900536737"} 2023-02-08 23:55:36,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-08 23:55:36,739 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-08 23:55:36,741 DEBUG [PEWorker-3] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-08 23:55:36,741 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, ASSIGN}] 2023-02-08 23:55:36,742 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, ASSIGN 2023-02-08 23:55:36,742 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, ASSIGN 2023-02-08 23:55:36,742 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, ASSIGN 2023-02-08 23:55:36,743 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-08 23:55:36,743 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-08 23:55:36,743 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-08 23:55:36,838 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:36,838 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:36,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-08 23:55:36,894 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=bf85808cb96a3fd70743466476b776c5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:36,894 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=c0432a17915eb080602d94742abd61bf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:36,894 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=83e0134f0861e917b7a0acc88b520e50, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:36,895 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900536894"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900536894"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900536894"}]},"ts":"1675900536894"} 2023-02-08 23:55:36,895 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900536894"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900536894"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900536894"}]},"ts":"1675900536894"} 2023-02-08 23:55:36,895 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900536894"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900536894"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900536894"}]},"ts":"1675900536894"} 2023-02-08 23:55:36,896 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=98, state=RUNNABLE; OpenRegionProcedure bf85808cb96a3fd70743466476b776c5, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:36,897 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=100, state=RUNNABLE; OpenRegionProcedure c0432a17915eb080602d94742abd61bf, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:36,897 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure 83e0134f0861e917b7a0acc88b520e50, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:37,039 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,039 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-08 23:55:37,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:37,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c0432a17915eb080602d94742abd61bf, NAME => 'testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 23:55:37,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:37,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:37,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:37,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:37,052 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:37,053 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1 2023-02-08 23:55:37,053 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1 2023-02-08 23:55:37,054 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0432a17915eb080602d94742abd61bf columnFamilyName family1 2023-02-08 23:55:37,072 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0e38670639294b138a863ec328b3514c 2023-02-08 23:55:37,078 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-388d3ec4cc4041828e2fe571247f5aad 2023-02-08 23:55:37,083 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-904a84fc1b8442109aa95d92a5281578 2023-02-08 23:55:37,087 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-d29e85ef6e604a50a4aa2d61a274e51f 2023-02-08 23:55:37,088 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(310): Store=c0432a17915eb080602d94742abd61bf/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:37,088 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:37,089 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2 2023-02-08 23:55:37,089 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2 2023-02-08 23:55:37,089 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0432a17915eb080602d94742abd61bf columnFamilyName family2 2023-02-08 23:55:37,095 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0725b9bf527c498bb05cd7b12e41b68a 2023-02-08 23:55:37,100 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-2d7243117fd7456eaf23062495841f44 2023-02-08 23:55:37,104 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-596a8c7e9ee74ee9a9c70737892c7b88 2023-02-08 23:55:37,109 DEBUG [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-73bfac9ee9854508b7540975ac4e8772 2023-02-08 23:55:37,109 INFO [StoreOpener-c0432a17915eb080602d94742abd61bf-1] regionserver.HStore(310): Store=c0432a17915eb080602d94742abd61bf/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:37,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:37,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:37,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:37,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c0432a17915eb080602d94742abd61bf 2023-02-08 23:55:37,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c0432a17915eb080602d94742abd61bf; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69680662, jitterRate=0.038322776556015015}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:37,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c0432a17915eb080602d94742abd61bf: Running coprocessor pre-open hook at 1675900537051Writing region info on filesystem at 1675900537051Initializing all the Stores at 1675900537052 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900537052Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900537052Cleaning up temporary data from old regions at 1675900537112 (+60 ms)Cleaning up detritus from prior splits at 1675900537112Running coprocessor post-open hooks at 1675900537115 (+3 ms)Region opened successfully at 1675900537115 2023-02-08 23:55:37,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf., pid=102, masterSystemTime=1675900537048 2023-02-08 23:55:37,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:37,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:37,116 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 23:55:37,116 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 23:55:37,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:37,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:37,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:37,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 83e0134f0861e917b7a0acc88b520e50, NAME => 'testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 23:55:37,118 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=c0432a17915eb080602d94742abd61bf, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:37,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:37,118 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900537118"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900537118"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900537118"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900537118"}]},"ts":"1675900537118"} 2023-02-08 23:55:37,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:37,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:37,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:37,119 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-08 23:55:37,119 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HStore(1912): c0432a17915eb080602d94742abd61bf/family2 is initiating minor compaction (all files) 2023-02-08 23:55:37,119 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-08 23:55:37,119 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c0432a17915eb080602d94742abd61bf/family2 in testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:37,119 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1912): c0432a17915eb080602d94742abd61bf/family1 is initiating minor compaction (all files) 2023-02-08 23:55:37,120 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c0432a17915eb080602d94742abd61bf/family1 in testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:55:37,120 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-2d7243117fd7456eaf23062495841f44, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-596a8c7e9ee74ee9a9c70737892c7b88, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-73bfac9ee9854508b7540975ac4e8772, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0725b9bf527c498bb05cd7b12e41b68a] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp, totalSize=20.4 K 2023-02-08 23:55:37,120 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0e38670639294b138a863ec328b3514c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-904a84fc1b8442109aa95d92a5281578, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-388d3ec4cc4041828e2fe571247f5aad, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-d29e85ef6e604a50a4aa2d61a274e51f] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp, totalSize=21.6 K 2023-02-08 23:55:37,121 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-2d7243117fd7456eaf23062495841f44, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512662 2023-02-08 23:55:37,121 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0e38670639294b138a863ec328b3514c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512662 2023-02-08 23:55:37,121 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:37,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=100 2023-02-08 23:55:37,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=100, state=SUCCESS; OpenRegionProcedure c0432a17915eb080602d94742abd61bf, server=jenkins-hbase4.apache.org,42703,1675900414768 in 223 msec 2023-02-08 23:55:37,122 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-596a8c7e9ee74ee9a9c70737892c7b88, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514449 2023-02-08 23:55:37,122 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-904a84fc1b8442109aa95d92a5281578, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514449 2023-02-08 23:55:37,122 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1 2023-02-08 23:55:37,122 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1 2023-02-08 23:55:37,122 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c0432a17915eb080602d94742abd61bf, ASSIGN in 380 msec 2023-02-08 23:55:37,123 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-73bfac9ee9854508b7540975ac4e8772, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515845 2023-02-08 23:55:37,123 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 83e0134f0861e917b7a0acc88b520e50 columnFamilyName family1 2023-02-08 23:55:37,123 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-388d3ec4cc4041828e2fe571247f5aad, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515845 2023-02-08 23:55:37,123 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0725b9bf527c498bb05cd7b12e41b68a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:37,124 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-d29e85ef6e604a50a4aa2d61a274e51f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:37,137 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-41e307b1667b4e89bec738267d5e3eb7 2023-02-08 23:55:37,145 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-8e816cbaac864e4bac1ad5c1403b2cda 2023-02-08 23:55:37,157 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-9589f8bc35fd4c6e98792433f593e54c 2023-02-08 23:55:37,165 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): c0432a17915eb080602d94742abd61bf#family1#compaction#113 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:37,168 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] throttle.PressureAwareThroughputController(145): c0432a17915eb080602d94742abd61bf#family2#compaction#114 average throughput is 0.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:37,174 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-efc80c1c81864c3990e1fec54dc689c0 2023-02-08 23:55:37,176 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(310): Store=83e0134f0861e917b7a0acc88b520e50/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:37,176 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:37,177 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2 2023-02-08 23:55:37,177 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2 2023-02-08 23:55:37,178 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 83e0134f0861e917b7a0acc88b520e50 columnFamilyName family2 2023-02-08 23:55:37,190 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-4461f76f0e1a4a289ede864a928e7ef2 2023-02-08 23:55:37,195 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-5fa5faab97ca4ee48dc3d414eaa6e9bb 2023-02-08 23:55:37,205 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7599740811c241538c4b650f4a4a7446 2023-02-08 23:55:37,211 DEBUG [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7a764657617342fe908e99ce225c7881 2023-02-08 23:55:37,211 INFO [StoreOpener-83e0134f0861e917b7a0acc88b520e50-1] regionserver.HStore(310): Store=83e0134f0861e917b7a0acc88b520e50/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:37,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:37,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:37,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:37,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:55:37,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 83e0134f0861e917b7a0acc88b520e50; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65485350, jitterRate=-0.024192243814468384}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:37,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 83e0134f0861e917b7a0acc88b520e50: Running coprocessor pre-open hook at 1675900537119Writing region info on filesystem at 1675900537119Initializing all the Stores at 1675900537120 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900537120Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900537120Cleaning up temporary data from old regions at 1675900537213 (+93 ms)Cleaning up detritus from prior splits at 1675900537214 (+1 ms)Running coprocessor post-open hooks at 1675900537218 (+4 ms)Region opened successfully at 1675900537218 2023-02-08 23:55:37,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50., pid=103, masterSystemTime=1675900537048 2023-02-08 23:55:37,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:55:37,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 23:55:37,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:37,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:37,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:37,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bf85808cb96a3fd70743466476b776c5, NAME => 'testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 23:55:37,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:37,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:37,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:37,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:37,221 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=83e0134f0861e917b7a0acc88b520e50, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:37,221 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900537221"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900537221"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900537221"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900537221"}]},"ts":"1675900537221"} 2023-02-08 23:55:37,222 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:37,223 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1 2023-02-08 23:55:37,223 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1 2023-02-08 23:55:37,223 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf85808cb96a3fd70743466476b776c5 columnFamilyName family1 2023-02-08 23:55:37,225 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-02-08 23:55:37,225 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure 83e0134f0861e917b7a0acc88b520e50, server=jenkins-hbase4.apache.org,42703,1675900414768 in 325 msec 2023-02-08 23:55:37,227 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=83e0134f0861e917b7a0acc88b520e50, ASSIGN in 484 msec 2023-02-08 23:55:37,234 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-97d9beddfbca474d9f178711cadcf3bd 2023-02-08 23:55:37,241 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-b18bc41b32274e03a92f7d9f409420b9 2023-02-08 23:55:37,248 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-be9073e70f064a84b2b0f1854b8db204 2023-02-08 23:55:37,256 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-ff75152bef214b51bf249e64e0341f34 2023-02-08 23:55:37,257 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(310): Store=bf85808cb96a3fd70743466476b776c5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:37,257 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:37,258 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2 2023-02-08 23:55:37,258 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2 2023-02-08 23:55:37,258 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf85808cb96a3fd70743466476b776c5 columnFamilyName family2 2023-02-08 23:55:37,271 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-20b322639109490b8cafc695fbc09c2f 2023-02-08 23:55:37,277 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-5e0aadad3d8e4d40a9a856d5d0e9935d 2023-02-08 23:55:37,283 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-c6f4612529414554b9acfc9c01140bb5 2023-02-08 23:55:37,290 DEBUG [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(539): loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-f6702cb56c6049bdb1dcc89a1b85895d 2023-02-08 23:55:37,290 INFO [StoreOpener-bf85808cb96a3fd70743466476b776c5-1] regionserver.HStore(310): Store=bf85808cb96a3fd70743466476b776c5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:37,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:37,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:37,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:37,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bf85808cb96a3fd70743466476b776c5 2023-02-08 23:55:37,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bf85808cb96a3fd70743466476b776c5; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58846240, jitterRate=-0.1231226921081543}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:37,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bf85808cb96a3fd70743466476b776c5: Running coprocessor pre-open hook at 1675900537221Writing region info on filesystem at 1675900537221Initializing all the Stores at 1675900537221Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900537221Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900537222 (+1 ms)Cleaning up temporary data from old regions at 1675900537292 (+70 ms)Cleaning up detritus from prior splits at 1675900537292Running coprocessor post-open hooks at 1675900537294 (+2 ms)Region opened successfully at 1675900537294 2023-02-08 23:55:37,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5., pid=101, masterSystemTime=1675900537048 2023-02-08 23:55:37,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-08 23:55:37,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-08 23:55:37,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:37,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:37,297 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=bf85808cb96a3fd70743466476b776c5, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:37,297 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675900537297"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900537297"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900537297"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900537297"}]},"ts":"1675900537297"} 2023-02-08 23:55:37,299 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=98 2023-02-08 23:55:37,299 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=98, state=SUCCESS; OpenRegionProcedure bf85808cb96a3fd70743466476b776c5, server=jenkins-hbase4.apache.org,42703,1675900414768 in 402 msec 2023-02-08 23:55:37,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=98, resume processing ppid=97 2023-02-08 23:55:37,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=bf85808cb96a3fd70743466476b776c5, ASSIGN in 558 msec 2023-02-08 23:55:37,301 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900537301"}]},"ts":"1675900537301"} 2023-02-08 23:55:37,302 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-08 23:55:37,303 INFO [PEWorker-3] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-08 23:55:37,305 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 573 msec 2023-02-08 23:55:37,340 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,340 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:37,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-08 23:55:37,341 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-08 23:55:37,352 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:37,360 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:37,368 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:37,369 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:37,377 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 23:55:37,393 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-08 23:55:37,393 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:37,393 DEBUG [Listener at localhost/41521] zookeeper.ReadOnlyZKClient(139): Connect 0x0e13a0e8 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:55:37,398 DEBUG [Listener at localhost/41521] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65d35e4b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:55:37,398 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:37,398 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:37,398 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:37,872 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208881f83ab7feb4edf92ca233b19c78804 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208881f83ab7feb4edf92ca233b19c78804 2023-02-08 23:55:37,873 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:37,885 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:55:37,889 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53148, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:55:37,894 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,894 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:37,894 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:55:37,897 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59700, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:55:37,904 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/decbf66d5acc4ef48c62804d6745a3b3 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-08 23:55:37,905 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/decbf66d5acc4ef48c62804d6745a3b3}] 2023-02-08 23:55:37,910 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/decbf66d5acc4ef48c62804d6745a3b3 for inclusion in c0432a17915eb080602d94742abd61bf/family1 2023-02-08 23:55:37,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-08 23:55:37,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 23:55:37,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:37,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/decbf66d5acc4ef48c62804d6745a3b3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7karq0dfq4jev5rrqu8quumuk5jg91ef5dqibvm9aim7hvtik4bvbcaosksoa04h/family1/decbf66d5acc4ef48c62804d6745a3b3 2023-02-08 23:55:37,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7karq0dfq4jev5rrqu8quumuk5jg91ef5dqibvm9aim7hvtik4bvbcaosksoa04h/family1/decbf66d5acc4ef48c62804d6745a3b3 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/21aa7a55c5b7464eb7f77795d14781f1_SeqId_34_ 2023-02-08 23:55:37,915 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7karq0dfq4jev5rrqu8quumuk5jg91ef5dqibvm9aim7hvtik4bvbcaosksoa04h/family1/decbf66d5acc4ef48c62804d6745a3b3 into c0432a17915eb080602d94742abd61bf/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/21aa7a55c5b7464eb7f77795d14781f1_SeqId_34_ - updating store file list. 2023-02-08 23:55:37,917 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/21aa7a55c5b7464eb7f77795d14781f1_SeqId_34_ into c0432a17915eb080602d94742abd61bf/family1 2023-02-08 23:55:37,918 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7karq0dfq4jev5rrqu8quumuk5jg91ef5dqibvm9aim7hvtik4bvbcaosksoa04h/family1/decbf66d5acc4ef48c62804d6745a3b3 into c0432a17915eb080602d94742abd61bf/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/21aa7a55c5b7464eb7f77795d14781f1_SeqId_34_) 2023-02-08 23:55:37,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__7karq0dfq4jev5rrqu8quumuk5jg91ef5dqibvm9aim7hvtik4bvbcaosksoa04h/family1/decbf66d5acc4ef48c62804d6745a3b3 2023-02-08 23:55:37,920 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:37,920 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:37,921 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302080854ba64b08e4a44a8ef84237326c925 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302080854ba64b08e4a44a8ef84237326c925 2023-02-08 23:55:37,922 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020824290db562504ae099dc7ac42a91428a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020824290db562504ae099dc7ac42a91428a 2023-02-08 23:55:37,923 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302082fa734f47d4b4337ace2dfaa02fb8c4a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302082fa734f47d4b4337ace2dfaa02fb8c4a 2023-02-08 23:55:37,924 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302084298f8536b12414885004ca6363d5751 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302084298f8536b12414885004ca6363d5751 2023-02-08 23:55:37,924 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 23:55:37,924 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 23:55:37,924 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:37,994 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family1/5ce6b291e1924e7da8a7a30a9ee5a8f1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/5ce6b291e1924e7da8a7a30a9ee5a8f1 2023-02-08 23:55:37,998 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-08 23:55:37,999 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c0432a17915eb080602d94742abd61bf/family1 of c0432a17915eb080602d94742abd61bf into 5ce6b291e1924e7da8a7a30a9ee5a8f1(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:37,999 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:37,999 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf., storeName=c0432a17915eb080602d94742abd61bf/family1, priority=12, startTime=1675900537115; duration=0sec 2023-02-08 23:55:37,999 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-08 23:55:38,000 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 23:55:38,002 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-08 23:55:38,002 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1912): 83e0134f0861e917b7a0acc88b520e50/family1 is initiating minor compaction (all files) 2023-02-08 23:55:38,002 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 83e0134f0861e917b7a0acc88b520e50/family1 in testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:38,002 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-efc80c1c81864c3990e1fec54dc689c0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-8e816cbaac864e4bac1ad5c1403b2cda, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-41e307b1667b4e89bec738267d5e3eb7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-9589f8bc35fd4c6e98792433f593e54c] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp, totalSize=21.6 K 2023-02-08 23:55:38,003 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-efc80c1c81864c3990e1fec54dc689c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512661 2023-02-08 23:55:38,004 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-8e816cbaac864e4bac1ad5c1403b2cda, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514449 2023-02-08 23:55:38,004 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-41e307b1667b4e89bec738267d5e3eb7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515845 2023-02-08 23:55:38,004 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-9589f8bc35fd4c6e98792433f593e54c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:38,020 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 83e0134f0861e917b7a0acc88b520e50#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:38,027 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0e38670639294b138a863ec328b3514c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-904a84fc1b8442109aa95d92a5281578, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-388d3ec4cc4041828e2fe571247f5aad, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-d29e85ef6e604a50a4aa2d61a274e51f] to archive 2023-02-08 23:55:38,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:38,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0e38670639294b138a863ec328b3514c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0e38670639294b138a863ec328b3514c 2023-02-08 23:55:38,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-904a84fc1b8442109aa95d92a5281578 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-904a84fc1b8442109aa95d92a5281578 2023-02-08 23:55:38,031 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-388d3ec4cc4041828e2fe571247f5aad to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-388d3ec4cc4041828e2fe571247f5aad 2023-02-08 23:55:38,032 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-d29e85ef6e604a50a4aa2d61a274e51f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family1/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-d29e85ef6e604a50a4aa2d61a274e51f 2023-02-08 23:55:38,267 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208713f9a042c604423965229b502107a0a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208713f9a042c604423965229b502107a0a 2023-02-08 23:55:38,268 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:38,269 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208b318e8e98c654ab69147a395a653f880 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208b318e8e98c654ab69147a395a653f880 2023-02-08 23:55:38,270 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:38,286 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,286 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,286 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,286 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,296 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/92c84a26b8954f4488ccf15b6b63df5e first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-08 23:55:38,296 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/75f17abab5364d5ba87017878570b480 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-08 23:55:38,297 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=32 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/92c84a26b8954f4488ccf15b6b63df5e}] 2023-02-08 23:55:38,297 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/75f17abab5364d5ba87017878570b480}] 2023-02-08 23:55:38,299 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/92c84a26b8954f4488ccf15b6b63df5e for inclusion in bf85808cb96a3fd70743466476b776c5/family1 2023-02-08 23:55:38,299 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/75f17abab5364d5ba87017878570b480 for inclusion in 83e0134f0861e917b7a0acc88b520e50/family1 2023-02-08 23:55:38,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-08 23:55:38,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last=1 2023-02-08 23:55:38,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-08 23:55:38,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-08 23:55:38,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:38,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:38,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/92c84a26b8954f4488ccf15b6b63df5e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__kq12nhrgs5rd0c0lgahhbkd15vgsnb16qvkjin9gilalo3ctlhs3je4o9pmqpin7/family1/92c84a26b8954f4488ccf15b6b63df5e 2023-02-08 23:55:38,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/75f17abab5364d5ba87017878570b480 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__tdlhj0kq201r3ektdo937rs0f02moeperkqq0kvlcl39kcijuthsp3cfpmavmcqo/family1/75f17abab5364d5ba87017878570b480 2023-02-08 23:55:38,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__kq12nhrgs5rd0c0lgahhbkd15vgsnb16qvkjin9gilalo3ctlhs3je4o9pmqpin7/family1/92c84a26b8954f4488ccf15b6b63df5e as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/7a661ed1f244456289f651b30f61d6c4_SeqId_34_ 2023-02-08 23:55:38,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__tdlhj0kq201r3ektdo937rs0f02moeperkqq0kvlcl39kcijuthsp3cfpmavmcqo/family1/75f17abab5364d5ba87017878570b480 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/c8f54fa430c04497bbd97924a6b77b66_SeqId_34_ 2023-02-08 23:55:38,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__kq12nhrgs5rd0c0lgahhbkd15vgsnb16qvkjin9gilalo3ctlhs3je4o9pmqpin7/family1/92c84a26b8954f4488ccf15b6b63df5e into bf85808cb96a3fd70743466476b776c5/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/7a661ed1f244456289f651b30f61d6c4_SeqId_34_ - updating store file list. 2023-02-08 23:55:38,306 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__tdlhj0kq201r3ektdo937rs0f02moeperkqq0kvlcl39kcijuthsp3cfpmavmcqo/family1/75f17abab5364d5ba87017878570b480 into 83e0134f0861e917b7a0acc88b520e50/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/c8f54fa430c04497bbd97924a6b77b66_SeqId_34_ - updating store file list. 2023-02-08 23:55:38,310 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/7a661ed1f244456289f651b30f61d6c4_SeqId_34_ into bf85808cb96a3fd70743466476b776c5/family1 2023-02-08 23:55:38,310 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__kq12nhrgs5rd0c0lgahhbkd15vgsnb16qvkjin9gilalo3ctlhs3je4o9pmqpin7/family1/92c84a26b8954f4488ccf15b6b63df5e into bf85808cb96a3fd70743466476b776c5/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/7a661ed1f244456289f651b30f61d6c4_SeqId_34_) 2023-02-08 23:55:38,310 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/c8f54fa430c04497bbd97924a6b77b66_SeqId_34_ into 83e0134f0861e917b7a0acc88b520e50/family1 2023-02-08 23:55:38,310 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__tdlhj0kq201r3ektdo937rs0f02moeperkqq0kvlcl39kcijuthsp3cfpmavmcqo/family1/75f17abab5364d5ba87017878570b480 into 83e0134f0861e917b7a0acc88b520e50/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/c8f54fa430c04497bbd97924a6b77b66_SeqId_34_) 2023-02-08 23:55:38,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__kq12nhrgs5rd0c0lgahhbkd15vgsnb16qvkjin9gilalo3ctlhs3je4o9pmqpin7/family1/92c84a26b8954f4488ccf15b6b63df5e 2023-02-08 23:55:38,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:38,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testCompactionWithHFileLink__tdlhj0kq201r3ektdo937rs0f02moeperkqq0kvlcl39kcijuthsp3cfpmavmcqo/family1/75f17abab5364d5ba87017878570b480 2023-02-08 23:55:38,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-08 23:55:38,313 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:38,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(1912): bf85808cb96a3fd70743466476b776c5/family1 is initiating minor compaction (all files) 2023-02-08 23:55:38,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@56272d58; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-08 23:55:38,314 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:38,315 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:38,315 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302084c94eb80abf24e1f920601f01f56cda5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302084c94eb80abf24e1f920601f01f56cda5 2023-02-08 23:55:38,316 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:38,317 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302086879d9029967492f881893b3ece54a05 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302086879d9029967492f881893b3ece54a05 2023-02-08 23:55:38,317 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023020845886db6ce904d17bca6173c2e44fa1c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023020845886db6ce904d17bca6173c2e44fa1c 2023-02-08 23:55:38,318 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020896ce51894cc442cbaafeb262785520c6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020896ce51894cc442cbaafeb262785520c6 2023-02-08 23:55:38,318 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302088eb3af42556a4def9b2293f1392ac0a3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302088eb3af42556a4def9b2293f1392ac0a3 2023-02-08 23:55:38,318 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230208eee36449625442d0980dcaeb5c877f3f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230208eee36449625442d0980dcaeb5c877f3f 2023-02-08 23:55:38,319 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 23:55:38,319 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208' 2023-02-08 23:55:38,319 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:38,319 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302088edfd6d65d9c4a9c9ada8e78ee01fb64 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302088edfd6d65d9c4a9c9ada8e78ee01fb64 2023-02-08 23:55:38,320 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230208b67eff96efab410aa112021b51ca7dfc to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230208b67eff96efab410aa112021b51ca7dfc 2023-02-08 23:55:38,320 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 23:55:38,320 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 23:55:38,321 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 23:55:38,321 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-08 23:55:38,323 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:38,326 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:38,330 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:38,331 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:38,337 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:38,338 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:38,355 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=282 (was 245) - Thread LEAK? -, OpenFileDescriptor=749 (was 619) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=118 (was 131), ProcessCount=168 (was 168), AvailableMemoryMB=3070 (was 3225) 2023-02-08 23:55:38,368 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=282, OpenFileDescriptor=749, MaxFileDescriptor=60000, SystemLoadAverage=118, ProcessCount=168, AvailableMemoryMB=3070 2023-02-08 23:55:38,369 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,369 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,373 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.HMaster$15(2997): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-02-08 23:55:38,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-08 23:55:38,378 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,378 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-08 23:55:38,380 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 23:55:38,384 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 10 msec 2023-02-08 23:55:38,394 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/.tmp/family2/12c70caadc5146c884780b2d9d6ccede as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/12c70caadc5146c884780b2d9d6ccede 2023-02-08 23:55:38,398 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c0432a17915eb080602d94742abd61bf/family2 of c0432a17915eb080602d94742abd61bf into 12c70caadc5146c884780b2d9d6ccede(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:38,398 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c0432a17915eb080602d94742abd61bf: 2023-02-08 23:55:38,398 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf., storeName=c0432a17915eb080602d94742abd61bf/family2, priority=12, startTime=1675900537116; duration=1sec 2023-02-08 23:55:38,398 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-08 23:55:38,398 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HRegion(2258): Starting compaction of bf85808cb96a3fd70743466476b776c5/family1 in testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:38,398 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-be9073e70f064a84b2b0f1854b8db204, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-ff75152bef214b51bf249e64e0341f34, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-97d9beddfbca474d9f178711cadcf3bd, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-b18bc41b32274e03a92f7d9f409420b9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/7a661ed1f244456289f651b30f61d6c4_SeqId_34_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp, totalSize=28.9 K 2023-02-08 23:55:38,399 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-be9073e70f064a84b2b0f1854b8db204, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512659 2023-02-08 23:55:38,400 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-ff75152bef214b51bf249e64e0341f34, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514448 2023-02-08 23:55:38,400 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-97d9beddfbca474d9f178711cadcf3bd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515844 2023-02-08 23:55:38,401 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-b18bc41b32274e03a92f7d9f409420b9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:38,401 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting 7a661ed1f244456289f651b30f61d6c4_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1675900512659 2023-02-08 23:55:38,413 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] throttle.PressureAwareThroughputController(145): bf85808cb96a3fd70743466476b776c5#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:38,432 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-2d7243117fd7456eaf23062495841f44, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-596a8c7e9ee74ee9a9c70737892c7b88, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-73bfac9ee9854508b7540975ac4e8772, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0725b9bf527c498bb05cd7b12e41b68a] to archive 2023-02-08 23:55:38,432 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:38,434 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-2d7243117fd7456eaf23062495841f44 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-2d7243117fd7456eaf23062495841f44 2023-02-08 23:55:38,435 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-596a8c7e9ee74ee9a9c70737892c7b88 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-596a8c7e9ee74ee9a9c70737892c7b88 2023-02-08 23:55:38,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-73bfac9ee9854508b7540975ac4e8772 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-73bfac9ee9854508b7540975ac4e8772 2023-02-08 23:55:38,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0725b9bf527c498bb05cd7b12e41b68a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/family2/testCompactionWithHFileLink=c0432a17915eb080602d94742abd61bf-0725b9bf527c498bb05cd7b12e41b68a 2023-02-08 23:55:38,479 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,479 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-08 23:55:38,480 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,480 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,481 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 23:55:38,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-08 23:55:38,483 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:55:38,483 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-08 23:55:38,483 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,483 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,483 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:55:38,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 23:55:38,485 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:38,486 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:38,486 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:38,486 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc empty. 2023-02-08 23:55:38,486 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7 empty. 2023-02-08 23:55:38,486 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3 empty. 2023-02-08 23:55:38,486 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:38,486 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:38,486 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:38,487 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-08 23:55:38,584 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,584 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 23:55:38,785 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:38,786 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:38,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 23:55:38,893 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:38,895 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => 539644099f8843cf0be59ce746f2f9e3, NAME => 'ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:55:38,895 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => 9246c8eb0029ec8f06f1acd496f0cde7, NAME => 'ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:55:38,895 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9d98bce82dc93804dedabab1ead06ebc, NAME => 'ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:55:39,087 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:39,087 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:39,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 23:55:39,183 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:55:39,230 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family1/ce8e94ae239949e996f2794ddefab590 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/ce8e94ae239949e996f2794ddefab590 2023-02-08 23:55:39,234 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in bf85808cb96a3fd70743466476b776c5/family1 of bf85808cb96a3fd70743466476b776c5 into ce8e94ae239949e996f2794ddefab590(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:39,234 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family1/bc9041f971eb44b9bcc09eb67051512c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/bc9041f971eb44b9bcc09eb67051512c 2023-02-08 23:55:39,234 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:39,235 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1675900538313; duration=0sec 2023-02-08 23:55:39,235 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-08 23:55:39,235 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 23:55:39,237 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-08 23:55:39,237 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HStore(1912): 83e0134f0861e917b7a0acc88b520e50/family2 is initiating minor compaction (all files) 2023-02-08 23:55:39,237 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 83e0134f0861e917b7a0acc88b520e50/family2 in testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:55:39,237 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-4461f76f0e1a4a289ede864a928e7ef2, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7a764657617342fe908e99ce225c7881, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7599740811c241538c4b650f4a4a7446, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-5fa5faab97ca4ee48dc3d414eaa6e9bb] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp, totalSize=20.4 K 2023-02-08 23:55:39,237 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-4461f76f0e1a4a289ede864a928e7ef2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512661 2023-02-08 23:55:39,238 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7a764657617342fe908e99ce225c7881, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514449 2023-02-08 23:55:39,238 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 83e0134f0861e917b7a0acc88b520e50/family1 of 83e0134f0861e917b7a0acc88b520e50 into bc9041f971eb44b9bcc09eb67051512c(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:39,238 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:39,238 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50., storeName=83e0134f0861e917b7a0acc88b520e50/family1, priority=12, startTime=1675900537218; duration=1sec 2023-02-08 23:55:39,238 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 23:55:39,238 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7599740811c241538c4b650f4a4a7446, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515845 2023-02-08 23:55:39,239 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 23:55:39,239 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-5fa5faab97ca4ee48dc3d414eaa6e9bb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:39,240 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-08 23:55:39,240 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1912): bf85808cb96a3fd70743466476b776c5/family2 is initiating minor compaction (all files) 2023-02-08 23:55:39,240 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bf85808cb96a3fd70743466476b776c5/family2 in testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:55:39,241 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-f6702cb56c6049bdb1dcc89a1b85895d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-5e0aadad3d8e4d40a9a856d5d0e9935d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-c6f4612529414554b9acfc9c01140bb5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-20b322639109490b8cafc695fbc09c2f] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp, totalSize=20.4 K 2023-02-08 23:55:39,241 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-f6702cb56c6049bdb1dcc89a1b85895d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900512659 2023-02-08 23:55:39,242 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-5e0aadad3d8e4d40a9a856d5d0e9935d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900514448 2023-02-08 23:55:39,242 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-c6f4612529414554b9acfc9c01140bb5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900515844 2023-02-08 23:55:39,243 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-20b322639109490b8cafc695fbc09c2f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900517631 2023-02-08 23:55:39,252 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] throttle.PressureAwareThroughputController(145): 83e0134f0861e917b7a0acc88b520e50#family2#compaction#117 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:39,257 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): bf85808cb96a3fd70743466476b776c5#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:39,303 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:39,303 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing 9d98bce82dc93804dedabab1ead06ebc, disabling compactions & flushes 2023-02-08 23:55:39,303 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:39,303 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:39,303 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. after waiting 0 ms 2023-02-08 23:55:39,303 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:39,304 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:39,304 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 9d98bce82dc93804dedabab1ead06ebc: Waiting for close lock at 1675900539303Disabling compacts and flushes for region at 1675900539303Disabling writes for close at 1675900539303Writing region close event to WAL at 1675900539303Closed at 1675900539303 2023-02-08 23:55:39,304 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:39,304 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing 9246c8eb0029ec8f06f1acd496f0cde7, disabling compactions & flushes 2023-02-08 23:55:39,304 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:39,304 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:39,305 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:39,305 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing 539644099f8843cf0be59ce746f2f9e3, disabling compactions & flushes 2023-02-08 23:55:39,305 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. after waiting 0 ms 2023-02-08 23:55:39,305 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:39,305 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:39,305 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:39,305 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:39,305 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. after waiting 0 ms 2023-02-08 23:55:39,305 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for 9246c8eb0029ec8f06f1acd496f0cde7: Waiting for close lock at 1675900539304Disabling compacts and flushes for region at 1675900539304Disabling writes for close at 1675900539305 (+1 ms)Writing region close event to WAL at 1675900539305Closed at 1675900539305 2023-02-08 23:55:39,305 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:39,305 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:39,305 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for 539644099f8843cf0be59ce746f2f9e3: Waiting for close lock at 1675900539305Disabling compacts and flushes for region at 1675900539305Disabling writes for close at 1675900539305Writing region close event to WAL at 1675900539305Closed at 1675900539305 2023-02-08 23:55:39,307 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:55:39,308 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675900539307"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900539307"}]},"ts":"1675900539307"} 2023-02-08 23:55:39,308 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675900539307"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900539307"}]},"ts":"1675900539307"} 2023-02-08 23:55:39,308 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675900539307"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900539307"}]},"ts":"1675900539307"} 2023-02-08 23:55:39,309 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 23:55:39,310 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:55:39,310 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900539310"}]},"ts":"1675900539310"} 2023-02-08 23:55:39,311 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-08 23:55:39,313 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9d98bce82dc93804dedabab1ead06ebc, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9246c8eb0029ec8f06f1acd496f0cde7, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=539644099f8843cf0be59ce746f2f9e3, ASSIGN}] 2023-02-08 23:55:39,315 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9d98bce82dc93804dedabab1ead06ebc, ASSIGN 2023-02-08 23:55:39,315 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=539644099f8843cf0be59ce746f2f9e3, ASSIGN 2023-02-08 23:55:39,315 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9246c8eb0029ec8f06f1acd496f0cde7, ASSIGN 2023-02-08 23:55:39,315 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9d98bce82dc93804dedabab1ead06ebc, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:55:39,315 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=539644099f8843cf0be59ce746f2f9e3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:55:39,315 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9246c8eb0029ec8f06f1acd496f0cde7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:55:39,328 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-efc80c1c81864c3990e1fec54dc689c0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-8e816cbaac864e4bac1ad5c1403b2cda, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-41e307b1667b4e89bec738267d5e3eb7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-9589f8bc35fd4c6e98792433f593e54c] to archive 2023-02-08 23:55:39,328 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:39,330 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-efc80c1c81864c3990e1fec54dc689c0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-efc80c1c81864c3990e1fec54dc689c0 2023-02-08 23:55:39,331 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-8e816cbaac864e4bac1ad5c1403b2cda to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-8e816cbaac864e4bac1ad5c1403b2cda 2023-02-08 23:55:39,332 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-41e307b1667b4e89bec738267d5e3eb7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-41e307b1667b4e89bec738267d5e3eb7 2023-02-08 23:55:39,333 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-9589f8bc35fd4c6e98792433f593e54c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family1/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-9589f8bc35fd4c6e98792433f593e54c 2023-02-08 23:55:39,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-be9073e70f064a84b2b0f1854b8db204, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-ff75152bef214b51bf249e64e0341f34, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-97d9beddfbca474d9f178711cadcf3bd, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-b18bc41b32274e03a92f7d9f409420b9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/7a661ed1f244456289f651b30f61d6c4_SeqId_34_] to archive 2023-02-08 23:55:39,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:39,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-be9073e70f064a84b2b0f1854b8db204 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-be9073e70f064a84b2b0f1854b8db204 2023-02-08 23:55:39,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-ff75152bef214b51bf249e64e0341f34 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-ff75152bef214b51bf249e64e0341f34 2023-02-08 23:55:39,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-97d9beddfbca474d9f178711cadcf3bd to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-97d9beddfbca474d9f178711cadcf3bd 2023-02-08 23:55:39,347 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-b18bc41b32274e03a92f7d9f409420b9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-b18bc41b32274e03a92f7d9f409420b9 2023-02-08 23:55:39,348 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/7a661ed1f244456289f651b30f61d6c4_SeqId_34_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family1/7a661ed1f244456289f651b30f61d6c4_SeqId_34_ 2023-02-08 23:55:39,469 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=539644099f8843cf0be59ce746f2f9e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:39,469 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=9246c8eb0029ec8f06f1acd496f0cde7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:39,469 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675900539469"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900539469"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900539469"}]},"ts":"1675900539469"} 2023-02-08 23:55:39,469 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675900539469"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900539469"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900539469"}]},"ts":"1675900539469"} 2023-02-08 23:55:39,469 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9d98bce82dc93804dedabab1ead06ebc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:39,469 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675900539469"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900539469"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900539469"}]},"ts":"1675900539469"} 2023-02-08 23:55:39,471 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure 539644099f8843cf0be59ce746f2f9e3, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:39,472 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure 9246c8eb0029ec8f06f1acd496f0cde7, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:39,472 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=106, state=RUNNABLE; OpenRegionProcedure 9d98bce82dc93804dedabab1ead06ebc, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:39,588 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:39,588 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:39,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 23:55:39,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:39,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 539644099f8843cf0be59ce746f2f9e3, NAME => 'ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 23:55:39,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:39,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:39,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:39,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:39,627 INFO [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:39,629 DEBUG [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1 2023-02-08 23:55:39,629 DEBUG [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1 2023-02-08 23:55:39,629 INFO [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 539644099f8843cf0be59ce746f2f9e3 columnFamilyName family1 2023-02-08 23:55:39,631 INFO [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] regionserver.HStore(310): Store=539644099f8843cf0be59ce746f2f9e3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:39,631 INFO [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:39,632 DEBUG [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2 2023-02-08 23:55:39,632 DEBUG [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2 2023-02-08 23:55:39,632 INFO [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 539644099f8843cf0be59ce746f2f9e3 columnFamilyName family2 2023-02-08 23:55:39,633 INFO [StoreOpener-539644099f8843cf0be59ce746f2f9e3-1] regionserver.HStore(310): Store=539644099f8843cf0be59ce746f2f9e3/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:39,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:39,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:39,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:39,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:55:39,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:39,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 539644099f8843cf0be59ce746f2f9e3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65291997, jitterRate=-0.02707342803478241}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:39,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 539644099f8843cf0be59ce746f2f9e3: Running coprocessor pre-open hook at 1675900539626Writing region info on filesystem at 1675900539626Initializing all the Stores at 1675900539627 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900539627Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900539627Cleaning up temporary data from old regions at 1675900539635 (+8 ms)Cleaning up detritus from prior splits at 1675900539635Running coprocessor post-open hooks at 1675900539638 (+3 ms)Region opened successfully at 1675900539638 2023-02-08 23:55:39,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3., pid=109, masterSystemTime=1675900539623 2023-02-08 23:55:39,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:39,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:39,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:39,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9d98bce82dc93804dedabab1ead06ebc, NAME => 'ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 23:55:39,640 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=539644099f8843cf0be59ce746f2f9e3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:39,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:39,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:39,641 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675900539640"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900539640"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900539640"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900539640"}]},"ts":"1675900539640"} 2023-02-08 23:55:39,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:39,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:39,642 INFO [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:39,643 DEBUG [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1 2023-02-08 23:55:39,643 DEBUG [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1 2023-02-08 23:55:39,643 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-02-08 23:55:39,643 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure 539644099f8843cf0be59ce746f2f9e3, server=jenkins-hbase4.apache.org,42703,1675900414768 in 171 msec 2023-02-08 23:55:39,643 INFO [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9d98bce82dc93804dedabab1ead06ebc columnFamilyName family1 2023-02-08 23:55:39,644 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=539644099f8843cf0be59ce746f2f9e3, ASSIGN in 330 msec 2023-02-08 23:55:39,644 INFO [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] regionserver.HStore(310): Store=9d98bce82dc93804dedabab1ead06ebc/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:39,644 INFO [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:39,645 DEBUG [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2 2023-02-08 23:55:39,645 DEBUG [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2 2023-02-08 23:55:39,646 INFO [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9d98bce82dc93804dedabab1ead06ebc columnFamilyName family2 2023-02-08 23:55:39,646 INFO [StoreOpener-9d98bce82dc93804dedabab1ead06ebc-1] regionserver.HStore(310): Store=9d98bce82dc93804dedabab1ead06ebc/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:39,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:39,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:39,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:39,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:55:39,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:39,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9d98bce82dc93804dedabab1ead06ebc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64095482, jitterRate=-0.04490289092063904}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:39,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9d98bce82dc93804dedabab1ead06ebc: Running coprocessor pre-open hook at 1675900539641Writing region info on filesystem at 1675900539641Initializing all the Stores at 1675900539641Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900539641Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900539641Cleaning up temporary data from old regions at 1675900539648 (+7 ms)Cleaning up detritus from prior splits at 1675900539648Running coprocessor post-open hooks at 1675900539652 (+4 ms)Region opened successfully at 1675900539652 2023-02-08 23:55:39,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc., pid=111, masterSystemTime=1675900539623 2023-02-08 23:55:39,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:39,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:39,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:39,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9246c8eb0029ec8f06f1acd496f0cde7, NAME => 'ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 23:55:39,654 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9d98bce82dc93804dedabab1ead06ebc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:39,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:39,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:39,654 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675900539654"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900539654"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900539654"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900539654"}]},"ts":"1675900539654"} 2023-02-08 23:55:39,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:39,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:39,655 INFO [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:39,656 DEBUG [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1 2023-02-08 23:55:39,656 DEBUG [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1 2023-02-08 23:55:39,656 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=106 2023-02-08 23:55:39,657 INFO [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9246c8eb0029ec8f06f1acd496f0cde7 columnFamilyName family1 2023-02-08 23:55:39,657 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=106, state=SUCCESS; OpenRegionProcedure 9d98bce82dc93804dedabab1ead06ebc, server=jenkins-hbase4.apache.org,42703,1675900414768 in 183 msec 2023-02-08 23:55:39,657 INFO [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] regionserver.HStore(310): Store=9246c8eb0029ec8f06f1acd496f0cde7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:39,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9d98bce82dc93804dedabab1ead06ebc, ASSIGN in 344 msec 2023-02-08 23:55:39,658 INFO [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:39,659 DEBUG [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2 2023-02-08 23:55:39,659 DEBUG [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2 2023-02-08 23:55:39,660 INFO [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9246c8eb0029ec8f06f1acd496f0cde7 columnFamilyName family2 2023-02-08 23:55:39,661 INFO [StoreOpener-9246c8eb0029ec8f06f1acd496f0cde7-1] regionserver.HStore(310): Store=9246c8eb0029ec8f06f1acd496f0cde7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:39,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:39,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:39,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:55:39,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:55:39,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:39,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9246c8eb0029ec8f06f1acd496f0cde7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62364945, jitterRate=-0.07068990170955658}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:55:39,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9246c8eb0029ec8f06f1acd496f0cde7: Running coprocessor pre-open hook at 1675900539654Writing region info on filesystem at 1675900539654Initializing all the Stores at 1675900539655 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900539655Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900539655Cleaning up temporary data from old regions at 1675900539663 (+8 ms)Cleaning up detritus from prior splits at 1675900539664 (+1 ms)Running coprocessor post-open hooks at 1675900539674 (+10 ms)Region opened successfully at 1675900539674 2023-02-08 23:55:39,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7., pid=110, masterSystemTime=1675900539623 2023-02-08 23:55:39,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:39,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:39,677 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=9246c8eb0029ec8f06f1acd496f0cde7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:39,677 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675900539677"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900539677"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900539677"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900539677"}]},"ts":"1675900539677"} 2023-02-08 23:55:39,679 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-02-08 23:55:39,680 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure 9246c8eb0029ec8f06f1acd496f0cde7, server=jenkins-hbase4.apache.org,42703,1675900414768 in 206 msec 2023-02-08 23:55:39,681 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=105 2023-02-08 23:55:39,681 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9246c8eb0029ec8f06f1acd496f0cde7, ASSIGN in 366 msec 2023-02-08 23:55:39,682 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:55:39,682 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900539682"}]},"ts":"1675900539682"} 2023-02-08 23:55:39,683 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-08 23:55:39,685 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:55:39,686 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2040 sec 2023-02-08 23:55:40,477 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/.tmp/family2/608e815748b447218dd115566f23a46c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/608e815748b447218dd115566f23a46c 2023-02-08 23:55:40,481 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/.tmp/family2/3fa1f72e0a9d47dd946cf6c1dbccd763 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/3fa1f72e0a9d47dd946cf6c1dbccd763 2023-02-08 23:55:40,482 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 83e0134f0861e917b7a0acc88b520e50/family2 of 83e0134f0861e917b7a0acc88b520e50 into 608e815748b447218dd115566f23a46c(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:40,482 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 83e0134f0861e917b7a0acc88b520e50: 2023-02-08 23:55:40,482 INFO [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50., storeName=83e0134f0861e917b7a0acc88b520e50/family2, priority=12, startTime=1675900537218; duration=1sec 2023-02-08 23:55:40,482 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 23:55:40,482 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 23:55:40,482 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-08 23:55:40,482 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-08 23:55:40,482 DEBUG [RS:0;jenkins-hbase4:42703-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. because compaction request was cancelled 2023-02-08 23:55:40,486 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in bf85808cb96a3fd70743466476b776c5/family2 of bf85808cb96a3fd70743466476b776c5 into 3fa1f72e0a9d47dd946cf6c1dbccd763(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:40,486 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf85808cb96a3fd70743466476b776c5: 2023-02-08 23:55:40,486 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5., storeName=bf85808cb96a3fd70743466476b776c5/family2, priority=12, startTime=1675900537295; duration=1sec 2023-02-08 23:55:40,486 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:55:40,523 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-4461f76f0e1a4a289ede864a928e7ef2, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7a764657617342fe908e99ce225c7881, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7599740811c241538c4b650f4a4a7446, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-5fa5faab97ca4ee48dc3d414eaa6e9bb] to archive 2023-02-08 23:55:40,524 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:40,525 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-4461f76f0e1a4a289ede864a928e7ef2 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-4461f76f0e1a4a289ede864a928e7ef2 2023-02-08 23:55:40,526 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7a764657617342fe908e99ce225c7881 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7a764657617342fe908e99ce225c7881 2023-02-08 23:55:40,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7599740811c241538c4b650f4a4a7446 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-7599740811c241538c4b650f4a4a7446 2023-02-08 23:55:40,528 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-5fa5faab97ca4ee48dc3d414eaa6e9bb to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/family2/testCompactionWithHFileLink=83e0134f0861e917b7a0acc88b520e50-5fa5faab97ca4ee48dc3d414eaa6e9bb 2023-02-08 23:55:40,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-f6702cb56c6049bdb1dcc89a1b85895d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-5e0aadad3d8e4d40a9a856d5d0e9935d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-c6f4612529414554b9acfc9c01140bb5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-20b322639109490b8cafc695fbc09c2f] to archive 2023-02-08 23:55:40,530 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:40,531 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-f6702cb56c6049bdb1dcc89a1b85895d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-f6702cb56c6049bdb1dcc89a1b85895d 2023-02-08 23:55:40,532 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-5e0aadad3d8e4d40a9a856d5d0e9935d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-5e0aadad3d8e4d40a9a856d5d0e9935d 2023-02-08 23:55:40,533 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-c6f4612529414554b9acfc9c01140bb5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-c6f4612529414554b9acfc9c01140bb5 2023-02-08 23:55:40,534 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-20b322639109490b8cafc695fbc09c2f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/family2/testCompactionWithHFileLink=bf85808cb96a3fd70743466476b776c5-20b322639109490b8cafc695fbc09c2f 2023-02-08 23:55:40,589 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:40,589 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:40,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 23:55:40,590 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-08 23:55:40,595 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:40,596 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:40,596 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:40,597 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:40,597 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:40,598 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:40,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 23:55:40,599 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 23:55:40,599 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:40,599 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 23:55:40,599 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 23:55:40,600 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:40,600 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:40,605 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:40,605 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:40,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:40,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:40,605 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:40,605 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:40,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:40,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 23:55:40,606 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 23:55:40,607 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 23:55:40,607 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:40,607 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 23:55:40,607 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:40,607 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. started... 2023-02-08 23:55:40,607 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:40,607 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:40,607 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:40,609 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 539644099f8843cf0be59ce746f2f9e3 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 23:55:40,609 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. started... 2023-02-08 23:55:40,607 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. started... 2023-02-08 23:55:40,609 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:40,609 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9246c8eb0029ec8f06f1acd496f0cde7 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 23:55:40,609 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9d98bce82dc93804dedabab1ead06ebc 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 23:55:41,020 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208f61416886a5c481b968b545444ff77b9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230208f61416886a5c481b968b545444ff77b9 2023-02-08 23:55:41,020 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302080315ab895e664b958006717392466084 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302080315ab895e664b958006717392466084 2023-02-08 23:55:41,021 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208b0c4ef14239f4fa69ca891e175a58484 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230208b0c4ef14239f4fa69ca891e175a58484 2023-02-08 23:55:41,425 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/a00d3474a0554ab6968f17669908b78a 2023-02-08 23:55:41,426 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/816a86b1241d43fe822b1171e1227da5 2023-02-08 23:55:41,427 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/45f8c1677678405c9832becbdccd6a4e 2023-02-08 23:55:41,844 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020862cb885e167a49aaa47213d053d94c6b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023020862cb885e167a49aaa47213d053d94c6b 2023-02-08 23:55:41,844 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302085be1a9c4289942219955eee0130afd58 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302085be1a9c4289942219955eee0130afd58 2023-02-08 23:55:41,844 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020804b28fc05139487ab7e96a2ffc7d4f61 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023020804b28fc05139487ab7e96a2ffc7d4f61 2023-02-08 23:55:41,934 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-08 23:55:42,249 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/630e65eb95704d27a9433b83eee4b79c 2023-02-08 23:55:42,249 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/e656241afc5e414cb19861b9674b6db2 2023-02-08 23:55:42,250 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/4b63be95dda64b7fbab51ee6ab2ca55e 2023-02-08 23:55:42,258 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/a00d3474a0554ab6968f17669908b78a as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/a00d3474a0554ab6968f17669908b78a 2023-02-08 23:55:42,260 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/45f8c1677678405c9832becbdccd6a4e as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/45f8c1677678405c9832becbdccd6a4e 2023-02-08 23:55:42,260 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/816a86b1241d43fe822b1171e1227da5 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/816a86b1241d43fe822b1171e1227da5 2023-02-08 23:55:42,263 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/a00d3474a0554ab6968f17669908b78a, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 23:55:42,264 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/45f8c1677678405c9832becbdccd6a4e, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 23:55:42,264 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/630e65eb95704d27a9433b83eee4b79c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/630e65eb95704d27a9433b83eee4b79c 2023-02-08 23:55:42,264 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/816a86b1241d43fe822b1171e1227da5, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 23:55:42,265 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/4b63be95dda64b7fbab51ee6ab2ca55e as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4b63be95dda64b7fbab51ee6ab2ca55e 2023-02-08 23:55:42,265 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/e656241afc5e414cb19861b9674b6db2 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/e656241afc5e414cb19861b9674b6db2 2023-02-08 23:55:42,277 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/e656241afc5e414cb19861b9674b6db2, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 23:55:42,277 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4b63be95dda64b7fbab51ee6ab2ca55e, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 23:55:42,277 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/630e65eb95704d27a9433b83eee4b79c, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 23:55:42,278 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 539644099f8843cf0be59ce746f2f9e3 in 1669ms, sequenceid=5, compaction requested=false 2023-02-08 23:55:42,278 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9246c8eb0029ec8f06f1acd496f0cde7 in 1669ms, sequenceid=5, compaction requested=false 2023-02-08 23:55:42,278 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 539644099f8843cf0be59ce746f2f9e3: 2023-02-08 23:55:42,278 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9246c8eb0029ec8f06f1acd496f0cde7: 2023-02-08 23:55:42,278 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:42,279 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:42,279 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9d98bce82dc93804dedabab1ead06ebc in 1670ms, sequenceid=5, compaction requested=false 2023-02-08 23:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:42,279 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9d98bce82dc93804dedabab1ead06ebc: 2023-02-08 23:55:42,279 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 23:55:42,279 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 23:55:42,281 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,281 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:42,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:42,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:42,281 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:42,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:42,281 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:42,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:42,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:42,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:42,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 23:55:42,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cffb5a8[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:42,283 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 23:55:42,283 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:42,284 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:42,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:42,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:42,284 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 23:55:42,284 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 23:55:42,284 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,284 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:42,284 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:42,286 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 23:55:42,286 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,286 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:42,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:42,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:42,286 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 23:55:42,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:42,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 23:55:42,289 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 23:55:42,289 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:42,289 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:42,289 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 23:55:42,289 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:42,292 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:42,292 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:42,292 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:42,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:42,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:42,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:42,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:42,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:42,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:42,292 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:42,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:42,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:42,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:42,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:42,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:42,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,294 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,296 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,296 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:42,296 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:42,296 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:42,296 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 23:55:42,296 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:42,296 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:42,296 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:42,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:42,296 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:42,297 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:42,297 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:42,297 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:42,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:42,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:42,297 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:42,297 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:42,397 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:42,397 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,397 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:42,400 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:42,400 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:42,401 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:42,403 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:42,403 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:42,403 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:42,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:42,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 23:55:42,405 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 23:55:42,405 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:42,405 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 23:55:42,405 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 23:55:42,406 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:42,406 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:42,407 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:42,407 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:42,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:42,408 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:42,408 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:42,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:42,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:42,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:42,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:42,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 23:55:42,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-08 23:55:42,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 23:55:42,409 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 23:55:42,409 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:42,409 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 23:55:42,409 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:42,410 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:42,410 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:42,410 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. started... 2023-02-08 23:55:42,410 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:42,410 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. started... 2023-02-08 23:55:42,410 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 539644099f8843cf0be59ce746f2f9e3 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 23:55:42,410 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. started... 2023-02-08 23:55:42,410 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9d98bce82dc93804dedabab1ead06ebc 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 23:55:42,410 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9246c8eb0029ec8f06f1acd496f0cde7 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 23:55:42,833 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302086c8d1835d033488ba91d7a6f2cb9752d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302086c8d1835d033488ba91d7a6f2cb9752d 2023-02-08 23:55:42,843 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208daccefd256374c8994802e8c1b18688a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230208daccefd256374c8994802e8c1b18688a 2023-02-08 23:55:42,853 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020884d5cf5faefc40be92896904fed6b09e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023020884d5cf5faefc40be92896904fed6b09e 2023-02-08 23:55:43,238 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/0927e87ec7f44df9aa6c9eaaad2a1e60 2023-02-08 23:55:43,247 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/7f9bd460450f40c3b55e63b309ff3eba 2023-02-08 23:55:43,257 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/10c96898e8614324b6a313e72422ab81 2023-02-08 23:55:43,651 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208238eba3e26aa4d019841d59f1e0760e2 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230208238eba3e26aa4d019841d59f1e0760e2 2023-02-08 23:55:43,659 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020828d6233959c6445e89fe10c2852830dd to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023020828d6233959c6445e89fe10c2852830dd 2023-02-08 23:55:43,668 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208452b3a6b933f4b8f8126ac92069da35b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230208452b3a6b933f4b8f8126ac92069da35b 2023-02-08 23:55:44,056 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/9b55e41a4ac9400aa7db075985870b2f 2023-02-08 23:55:44,060 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/0927e87ec7f44df9aa6c9eaaad2a1e60 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/0927e87ec7f44df9aa6c9eaaad2a1e60 2023-02-08 23:55:44,063 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/0927e87ec7f44df9aa6c9eaaad2a1e60, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 23:55:44,063 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/053635334e9b41d1875e1fe3b0b547ff 2023-02-08 23:55:44,064 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/9b55e41a4ac9400aa7db075985870b2f as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/9b55e41a4ac9400aa7db075985870b2f 2023-02-08 23:55:44,067 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/9b55e41a4ac9400aa7db075985870b2f, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 23:55:44,068 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/7f9bd460450f40c3b55e63b309ff3eba as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/7f9bd460450f40c3b55e63b309ff3eba 2023-02-08 23:55:44,068 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9246c8eb0029ec8f06f1acd496f0cde7 in 1658ms, sequenceid=9, compaction requested=false 2023-02-08 23:55:44,068 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9246c8eb0029ec8f06f1acd496f0cde7: 2023-02-08 23:55:44,068 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:44,068 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:44,072 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/270ed97bb77f422fbaf9d8d18f5187ed 2023-02-08 23:55:44,074 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/7f9bd460450f40c3b55e63b309ff3eba, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 23:55:44,075 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/053635334e9b41d1875e1fe3b0b547ff as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/053635334e9b41d1875e1fe3b0b547ff 2023-02-08 23:55:44,077 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/10c96898e8614324b6a313e72422ab81 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/10c96898e8614324b6a313e72422ab81 2023-02-08 23:55:44,079 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/053635334e9b41d1875e1fe3b0b547ff, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 23:55:44,084 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9d98bce82dc93804dedabab1ead06ebc in 1674ms, sequenceid=9, compaction requested=false 2023-02-08 23:55:44,084 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9d98bce82dc93804dedabab1ead06ebc: 2023-02-08 23:55:44,084 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:44,084 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:44,086 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/10c96898e8614324b6a313e72422ab81, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 23:55:44,087 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/270ed97bb77f422fbaf9d8d18f5187ed as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/270ed97bb77f422fbaf9d8d18f5187ed 2023-02-08 23:55:44,091 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/270ed97bb77f422fbaf9d8d18f5187ed, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 23:55:44,092 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 539644099f8843cf0be59ce746f2f9e3 in 1682ms, sequenceid=9, compaction requested=false 2023-02-08 23:55:44,092 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 539644099f8843cf0be59ce746f2f9e3: 2023-02-08 23:55:44,092 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:44,092 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:44,092 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:44,092 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,092 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 23:55:44,093 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 23:55:44,094 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:44,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:44,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:44,094 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:44,094 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:44,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:44,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:44,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:44,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:44,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 23:55:44,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7081d303[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:44,096 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 23:55:44,096 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:44,097 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:44,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:44,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:44,097 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 23:55:44,097 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 23:55:44,097 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:44,097 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,097 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:44,102 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 23:55:44,102 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,102 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:44,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:44,102 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 23:55:44,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:44,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:44,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:44,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:44,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 23:55:44,104 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 23:55:44,104 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:44,104 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:44,105 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 23:55:44,105 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:44,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:44,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:44,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:44,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:44,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:44,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:44,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:44,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:44,106 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:44,107 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,109 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,109 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:44,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,109 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 23:55:44,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-08 23:55:44,109 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,109 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:44,109 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,110 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:44,110 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:44,110 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,110 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:44,110 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:44,110 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:44,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:44,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:44,110 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:44,110 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:44,210 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:44,210 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:44,210 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:44,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:44,212 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:44,212 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:44,213 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:44,214 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:44,214 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:44,215 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:44,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:44,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 23:55:44,216 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 23:55:44,216 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,216 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 23:55:44,216 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 23:55:44,217 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:44,217 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:44,219 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,219 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:44,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:44,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:44,219 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:44,219 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:44,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:44,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:44,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:44,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:44,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 23:55:44,220 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-08 23:55:44,220 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 23:55:44,220 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 23:55:44,220 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:44,220 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 23:55:44,220 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:44,220 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:44,220 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:44,220 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:44,220 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. started... 2023-02-08 23:55:44,221 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. started... 2023-02-08 23:55:44,220 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:44,220 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. started... 2023-02-08 23:55:44,221 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9d98bce82dc93804dedabab1ead06ebc 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 23:55:44,221 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9246c8eb0029ec8f06f1acd496f0cde7 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 23:55:44,221 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 539644099f8843cf0be59ce746f2f9e3 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 23:55:44,233 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208872fb38d48c7445e99dab8e0057d47c6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230208872fb38d48c7445e99dab8e0057d47c6 2023-02-08 23:55:44,238 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/580ad38528c044c882078798ed8c7afb 2023-02-08 23:55:44,634 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208b7f83daff52a45668505d58925e0b3d1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230208b7f83daff52a45668505d58925e0b3d1 2023-02-08 23:55:44,635 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302083196b4e4d3074a4888a867b3db32e847 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302083196b4e4d3074a4888a867b3db32e847 2023-02-08 23:55:44,641 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/8ddb16124e6b49f99bfdeb1a415b184e 2023-02-08 23:55:44,649 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208c187c826715b4ff4bdbb2aa3c474f35d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230208c187c826715b4ff4bdbb2aa3c474f35d 2023-02-08 23:55:44,665 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/003d63aa5bfc41d794c4dd002f343159 2023-02-08 23:55:44,669 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/580ad38528c044c882078798ed8c7afb as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/580ad38528c044c882078798ed8c7afb 2023-02-08 23:55:44,672 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/580ad38528c044c882078798ed8c7afb, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 23:55:44,673 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/003d63aa5bfc41d794c4dd002f343159 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/003d63aa5bfc41d794c4dd002f343159 2023-02-08 23:55:44,677 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/003d63aa5bfc41d794c4dd002f343159, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 23:55:44,678 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 539644099f8843cf0be59ce746f2f9e3 in 457ms, sequenceid=13, compaction requested=true 2023-02-08 23:55:44,678 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 539644099f8843cf0be59ce746f2f9e3: 2023-02-08 23:55:44,678 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:44,678 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:45,039 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/667901d58f15419e92c54106c66bfc09 2023-02-08 23:55:45,067 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208b11766c31edd4ae7a0d8cab34e433905 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230208b11766c31edd4ae7a0d8cab34e433905 2023-02-08 23:55:45,128 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:55:45,451 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020882b16c2c60474dab8c98e7378f6846d3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023020882b16c2c60474dab8c98e7378f6846d3 2023-02-08 23:55:45,473 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/85a66618c02546ec8241d11dbe24a284 2023-02-08 23:55:45,476 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/8ddb16124e6b49f99bfdeb1a415b184e as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/8ddb16124e6b49f99bfdeb1a415b184e 2023-02-08 23:55:45,479 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/8ddb16124e6b49f99bfdeb1a415b184e, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 23:55:45,480 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/85a66618c02546ec8241d11dbe24a284 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/85a66618c02546ec8241d11dbe24a284 2023-02-08 23:55:45,483 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/85a66618c02546ec8241d11dbe24a284, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 23:55:45,484 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9d98bce82dc93804dedabab1ead06ebc in 1263ms, sequenceid=13, compaction requested=true 2023-02-08 23:55:45,484 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9d98bce82dc93804dedabab1ead06ebc: 2023-02-08 23:55:45,484 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:45,484 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:45,856 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/4117d092636743c68da983412024d64e 2023-02-08 23:55:45,860 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/667901d58f15419e92c54106c66bfc09 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/667901d58f15419e92c54106c66bfc09 2023-02-08 23:55:45,863 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/667901d58f15419e92c54106c66bfc09, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 23:55:45,864 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/4117d092636743c68da983412024d64e as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4117d092636743c68da983412024d64e 2023-02-08 23:55:45,867 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4117d092636743c68da983412024d64e, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 23:55:45,867 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9246c8eb0029ec8f06f1acd496f0cde7 in 1646ms, sequenceid=13, compaction requested=true 2023-02-08 23:55:45,867 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9246c8eb0029ec8f06f1acd496f0cde7: 2023-02-08 23:55:45,868 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 23:55:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 23:55:45,869 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:45,869 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:45,870 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:45,870 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:45,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:45,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 23:55:45,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@517ca692[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:45,871 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 23:55:45,871 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:45,872 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:45,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:45,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:45,872 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 23:55:45,872 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 23:55:45,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:45,872 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 23:55:45,873 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 23:55:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:45,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 23:55:45,875 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 23:55:45,875 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:45,875 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:45,875 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 23:55:45,875 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:45,877 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:45,877 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:45,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:45,878 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:45,878 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:45,879 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:45,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:45,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:45,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:45,881 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:45,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:45,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 23:55:45,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:45,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:45,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-08 23:55:45,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:45,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:45,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:45,882 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:45,882 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:45,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:45,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:45,982 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:45,982 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,982 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:45,984 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:45,984 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:45,984 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:55:45,986 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:45,986 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:45,986 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:45,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:45,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 23:55:45,988 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 23:55:45,988 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:45,988 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 23:55:45,988 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 23:55:45,988 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:45,988 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:45,989 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,989 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:45,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:45,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:45,990 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:45,990 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:45,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:45,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:45,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:45,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:45,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 23:55:45,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-08 23:55:45,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 23:55:45,991 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 23:55:45,991 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:45,991 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 23:55:45,991 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:45,991 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. started... 2023-02-08 23:55:45,991 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:45,991 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. started... 2023-02-08 23:55:45,991 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 539644099f8843cf0be59ce746f2f9e3 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 23:55:45,991 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:45,991 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9d98bce82dc93804dedabab1ead06ebc 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 23:55:45,991 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:45,991 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:45,991 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. started... 2023-02-08 23:55:45,992 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9246c8eb0029ec8f06f1acd496f0cde7 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 23:55:46,007 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020824dc5d4a44d048b794441486a5ca0266 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023020824dc5d4a44d048b794441486a5ca0266 2023-02-08 23:55:46,405 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208451f309ffda540e6960fc2aaa28c99b2 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230208451f309ffda540e6960fc2aaa28c99b2 2023-02-08 23:55:46,408 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208a08ba2814d16460891109880961e86d0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230208a08ba2814d16460891109880961e86d0 2023-02-08 23:55:46,411 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/4087fff1c9dd4948908faee98d380124 2023-02-08 23:55:46,810 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/47f7697e55784f9290cd96b0f99dac54 2023-02-08 23:55:46,812 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/eca6ee43691b4609b35b57bb27100283 2023-02-08 23:55:46,823 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020860c57f142a854403bd0f0d2f1b35e82c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023020860c57f142a854403bd0f0d2f1b35e82c 2023-02-08 23:55:47,224 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208ad9de44703c14487b663b374daf063f9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230208ad9de44703c14487b663b374daf063f9 2023-02-08 23:55:47,226 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208753add58039d4155a1aef46cfd779a83 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230208753add58039d4155a1aef46cfd779a83 2023-02-08 23:55:47,229 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/8bb63c6dde544f8d8ca36ef866fcce14 2023-02-08 23:55:47,230 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/16e018eeea794fa4aedd9f090ad09075 2023-02-08 23:55:47,232 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/4087fff1c9dd4948908faee98d380124 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/4087fff1c9dd4948908faee98d380124 2023-02-08 23:55:47,233 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/eca6ee43691b4609b35b57bb27100283 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/eca6ee43691b4609b35b57bb27100283 2023-02-08 23:55:47,235 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/4087fff1c9dd4948908faee98d380124, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 23:55:47,236 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/8bb63c6dde544f8d8ca36ef866fcce14 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/8bb63c6dde544f8d8ca36ef866fcce14 2023-02-08 23:55:47,236 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/eca6ee43691b4609b35b57bb27100283, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 23:55:47,237 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/16e018eeea794fa4aedd9f090ad09075 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/16e018eeea794fa4aedd9f090ad09075 2023-02-08 23:55:47,240 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/8bb63c6dde544f8d8ca36ef866fcce14, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 23:55:47,240 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/16e018eeea794fa4aedd9f090ad09075, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 23:55:47,240 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9d98bce82dc93804dedabab1ead06ebc in 1249ms, sequenceid=17, compaction requested=true 2023-02-08 23:55:47,240 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9d98bce82dc93804dedabab1ead06ebc: 2023-02-08 23:55:47,240 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:47,240 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 539644099f8843cf0be59ce746f2f9e3 in 1249ms, sequenceid=17, compaction requested=true 2023-02-08 23:55:47,241 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:47,241 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 539644099f8843cf0be59ce746f2f9e3: 2023-02-08 23:55:47,241 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:47,241 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:47,628 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/bdfaacff8a0d459ca0e5d57f13a693a1 2023-02-08 23:55:47,633 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/47f7697e55784f9290cd96b0f99dac54 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/47f7697e55784f9290cd96b0f99dac54 2023-02-08 23:55:47,637 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/47f7697e55784f9290cd96b0f99dac54, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 23:55:47,638 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/bdfaacff8a0d459ca0e5d57f13a693a1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/bdfaacff8a0d459ca0e5d57f13a693a1 2023-02-08 23:55:47,643 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/bdfaacff8a0d459ca0e5d57f13a693a1, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 23:55:47,643 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9246c8eb0029ec8f06f1acd496f0cde7 in 1651ms, sequenceid=17, compaction requested=true 2023-02-08 23:55:47,644 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9246c8eb0029ec8f06f1acd496f0cde7: 2023-02-08 23:55:47,644 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:47,644 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:47,644 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:47,644 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,644 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 23:55:47,644 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 23:55:47,645 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:47,645 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,646 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:47,646 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:47,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 23:55:47,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a75f072[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:47,647 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 23:55:47,647 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:47,649 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:47,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:47,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:47,649 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 23:55:47,649 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 23:55:47,649 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:47,649 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,649 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:47,651 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 23:55:47,651 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,651 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,651 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 23:55:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:47,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:47,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 23:55:47,654 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 23:55:47,654 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:47,654 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:47,654 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 23:55:47,654 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:47,655 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:47,655 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:47,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:47,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:47,656 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:47,656 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:47,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:47,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:47,657 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:47,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:47,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:47,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:47,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,659 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 23:55:47,659 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:47,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:47,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:47,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:47,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:47,659 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:47,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,660 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:47,660 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:47,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-08 23:55:47,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:47,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:47,760 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:47,760 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,760 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:47,772 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:47,781 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:47,789 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:47,789 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:47,817 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:47,817 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:47,818 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:47,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:47,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 23:55:47,821 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 23:55:47,821 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,821 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 23:55:47,821 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 23:55:47,822 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:47,822 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:47,824 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,824 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:47,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:47,824 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:47,824 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:47,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:47,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 23:55:47,828 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-08 23:55:47,828 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 23:55:47,828 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 23:55:47,828 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:47,828 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 23:55:47,832 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:47,832 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. started... 2023-02-08 23:55:47,832 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:47,832 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:55:47,832 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 539644099f8843cf0be59ce746f2f9e3 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 23:55:47,832 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:47,832 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. started... 2023-02-08 23:55:47,832 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:47,832 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 9d98bce82dc93804dedabab1ead06ebc 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 23:55:47,832 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. started... 2023-02-08 23:55:47,832 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9246c8eb0029ec8f06f1acd496f0cde7 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 23:55:48,679 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/30efa72c82f144a7ad799117ae62c087 2023-02-08 23:55:48,679 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/fa86b37d11de4e6194560afffb5a23de 2023-02-08 23:55:48,680 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/ff704c6b0f28411cb605e2b9e6e01467 2023-02-08 23:55:48,683 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 30efa72c82f144a7ad799117ae62c087 2023-02-08 23:55:48,683 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa86b37d11de4e6194560afffb5a23de 2023-02-08 23:55:48,683 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff704c6b0f28411cb605e2b9e6e01467 2023-02-08 23:55:49,500 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/536ac926736f422ea3968d6b7f62d8f5 2023-02-08 23:55:49,501 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/5d2fdb45255849748b66c2604e916be2 2023-02-08 23:55:49,504 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 536ac926736f422ea3968d6b7f62d8f5 2023-02-08 23:55:49,504 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/bb6034ca556048aebe308d070f5a5675 2023-02-08 23:55:49,504 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d2fdb45255849748b66c2604e916be2 2023-02-08 23:55:49,505 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/ff704c6b0f28411cb605e2b9e6e01467 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/ff704c6b0f28411cb605e2b9e6e01467 2023-02-08 23:55:49,505 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/fa86b37d11de4e6194560afffb5a23de as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/fa86b37d11de4e6194560afffb5a23de 2023-02-08 23:55:49,507 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bb6034ca556048aebe308d070f5a5675 2023-02-08 23:55:49,508 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/30efa72c82f144a7ad799117ae62c087 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/30efa72c82f144a7ad799117ae62c087 2023-02-08 23:55:49,508 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa86b37d11de4e6194560afffb5a23de 2023-02-08 23:55:49,508 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/fa86b37d11de4e6194560afffb5a23de, entries=3, sequenceid=23, filesize=5.0 K 2023-02-08 23:55:49,508 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff704c6b0f28411cb605e2b9e6e01467 2023-02-08 23:55:49,508 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/ff704c6b0f28411cb605e2b9e6e01467, entries=3, sequenceid=23, filesize=5.0 K 2023-02-08 23:55:49,509 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/5d2fdb45255849748b66c2604e916be2 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/5d2fdb45255849748b66c2604e916be2 2023-02-08 23:55:49,509 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/536ac926736f422ea3968d6b7f62d8f5 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/536ac926736f422ea3968d6b7f62d8f5 2023-02-08 23:55:49,512 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 30efa72c82f144a7ad799117ae62c087 2023-02-08 23:55:49,512 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/30efa72c82f144a7ad799117ae62c087, entries=3, sequenceid=23, filesize=5.0 K 2023-02-08 23:55:49,512 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d2fdb45255849748b66c2604e916be2 2023-02-08 23:55:49,512 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/5d2fdb45255849748b66c2604e916be2, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 23:55:49,512 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 536ac926736f422ea3968d6b7f62d8f5 2023-02-08 23:55:49,513 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/536ac926736f422ea3968d6b7f62d8f5, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 23:55:49,513 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/bb6034ca556048aebe308d070f5a5675 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/bb6034ca556048aebe308d070f5a5675 2023-02-08 23:55:49,513 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9d98bce82dc93804dedabab1ead06ebc in 1681ms, sequenceid=23, compaction requested=true 2023-02-08 23:55:49,513 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 9d98bce82dc93804dedabab1ead06ebc: 2023-02-08 23:55:49,513 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:49,513 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:55:49,513 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9246c8eb0029ec8f06f1acd496f0cde7 in 1681ms, sequenceid=23, compaction requested=true 2023-02-08 23:55:49,513 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9246c8eb0029ec8f06f1acd496f0cde7: 2023-02-08 23:55:49,513 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:49,513 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:55:49,519 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bb6034ca556048aebe308d070f5a5675 2023-02-08 23:55:49,519 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/bb6034ca556048aebe308d070f5a5675, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 23:55:49,519 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 539644099f8843cf0be59ce746f2f9e3 in 1687ms, sequenceid=23, compaction requested=true 2023-02-08 23:55:49,519 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 539644099f8843cf0be59ce746f2f9e3: 2023-02-08 23:55:49,519 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:49,519 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:55:49,520 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:55:49,520 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,520 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 23:55:49,520 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 23:55:49,521 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:49,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:49,521 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:49,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:49,521 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:49,522 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:49,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:49,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:49,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:49,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 23:55:49,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ba65c61[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:49,523 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 23:55:49,523 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:49,524 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:49,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:49,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:49,524 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 23:55:49,524 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 23:55:49,524 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,524 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:49,524 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:49,529 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,529 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 23:55:49,529 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:49,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:49,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:49,529 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 23:55:49,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:49,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:49,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:49,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:49,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:49,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:49,531 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 23:55:49,531 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 23:55:49,531 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:49,531 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:49,531 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 23:55:49,531 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:49,533 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:49,533 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:49,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:49,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:49,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:49,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:49,533 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:49,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:49,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:49,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:49,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:49,533 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:49,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:49,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:49,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:49,534 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:49,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 23:55:49,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,536 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,536 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:49,536 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 23:55:49,536 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:49,536 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:49,536 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:49,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:49,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-08 23:55:49,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:49,536 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:49,536 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 23:55:49,537 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:49,536 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:49,537 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 23:55:49,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:49,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:49,537 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 23:55:49,537 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:49,537 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:49,637 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:49,637 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:49,638 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:49,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:49,638 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:49,638 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): 539644099f8843cf0be59ce746f2f9e3/family1 is initiating major compaction (all files) 2023-02-08 23:55:49,639 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:49,639 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:49,639 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of 539644099f8843cf0be59ce746f2f9e3/family1 in ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:49,639 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/816a86b1241d43fe822b1171e1227da5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/10c96898e8614324b6a313e72422ab81, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/580ad38528c044c882078798ed8c7afb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/eca6ee43691b4609b35b57bb27100283, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/30efa72c82f144a7ad799117ae62c087] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp, totalSize=26.6 K 2023-02-08 23:55:49,639 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 816a86b1241d43fe822b1171e1227da5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900540596 2023-02-08 23:55:49,640 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 10c96898e8614324b6a313e72422ab81, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900542401 2023-02-08 23:55:49,640 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 580ad38528c044c882078798ed8c7afb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900544213 2023-02-08 23:55:49,640 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting eca6ee43691b4609b35b57bb27100283, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900545984 2023-02-08 23:55:49,641 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 30efa72c82f144a7ad799117ae62c087, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:55:49,649 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): 539644099f8843cf0be59ce746f2f9e3#family1#compaction#149 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:50,057 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020887314bb284844b9d8889a36e5d8afffb_del 2023-02-08 23:55:50,057 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020887314bb284844b9d8889a36e5d8afffb_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023020887314bb284844b9d8889a36e5d8afffb_del 2023-02-08 23:55:50,869 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e86708f5e48c4e2092a01878fec80b37 2023-02-08 23:55:50,870 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family1/e86708f5e48c4e2092a01878fec80b37 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/e86708f5e48c4e2092a01878fec80b37 2023-02-08 23:55:50,873 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e86708f5e48c4e2092a01878fec80b37 2023-02-08 23:55:50,874 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 539644099f8843cf0be59ce746f2f9e3/family1 of 539644099f8843cf0be59ce746f2f9e3 into e86708f5e48c4e2092a01878fec80b37(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:50,874 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for 539644099f8843cf0be59ce746f2f9e3: 2023-02-08 23:55:50,874 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:50,874 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): 539644099f8843cf0be59ce746f2f9e3/family2 is initiating major compaction (all files) 2023-02-08 23:55:50,874 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:50,874 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:50,874 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of 539644099f8843cf0be59ce746f2f9e3/family2 in ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:55:50,875 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/e656241afc5e414cb19861b9674b6db2, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/270ed97bb77f422fbaf9d8d18f5187ed, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/003d63aa5bfc41d794c4dd002f343159, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/16e018eeea794fa4aedd9f090ad09075, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/bb6034ca556048aebe308d070f5a5675] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp, totalSize=25.3 K 2023-02-08 23:55:50,875 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting e656241afc5e414cb19861b9674b6db2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900540596 2023-02-08 23:55:50,875 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 270ed97bb77f422fbaf9d8d18f5187ed, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900542401 2023-02-08 23:55:50,875 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 003d63aa5bfc41d794c4dd002f343159, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900544213 2023-02-08 23:55:50,876 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 16e018eeea794fa4aedd9f090ad09075, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900545984 2023-02-08 23:55:50,876 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting bb6034ca556048aebe308d070f5a5675, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:55:50,883 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): 539644099f8843cf0be59ce746f2f9e3#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:50,923 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/816a86b1241d43fe822b1171e1227da5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/10c96898e8614324b6a313e72422ab81, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/580ad38528c044c882078798ed8c7afb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/eca6ee43691b4609b35b57bb27100283, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/30efa72c82f144a7ad799117ae62c087] to archive 2023-02-08 23:55:50,924 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:50,925 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/816a86b1241d43fe822b1171e1227da5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/816a86b1241d43fe822b1171e1227da5 2023-02-08 23:55:50,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/10c96898e8614324b6a313e72422ab81 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/10c96898e8614324b6a313e72422ab81 2023-02-08 23:55:50,927 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/580ad38528c044c882078798ed8c7afb to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/580ad38528c044c882078798ed8c7afb 2023-02-08 23:55:50,928 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/eca6ee43691b4609b35b57bb27100283 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/eca6ee43691b4609b35b57bb27100283 2023-02-08 23:55:50,929 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/30efa72c82f144a7ad799117ae62c087 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/30efa72c82f144a7ad799117ae62c087 2023-02-08 23:55:51,291 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230208cd718c89fe4e4b5e898dff28d32b566f_del 2023-02-08 23:55:51,291 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208cd718c89fe4e4b5e898dff28d32b566f_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230208cd718c89fe4e4b5e898dff28d32b566f_del 2023-02-08 23:55:52,104 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 140af52081274f49a8daf7a9e9052449 2023-02-08 23:55:52,105 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/.tmp/family2/140af52081274f49a8daf7a9e9052449 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/140af52081274f49a8daf7a9e9052449 2023-02-08 23:55:52,108 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 140af52081274f49a8daf7a9e9052449 2023-02-08 23:55:52,109 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 539644099f8843cf0be59ce746f2f9e3/family2 of 539644099f8843cf0be59ce746f2f9e3 into 140af52081274f49a8daf7a9e9052449(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:55:52,109 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for 539644099f8843cf0be59ce746f2f9e3: 2023-02-08 23:55:52,109 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:52,109 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): 9d98bce82dc93804dedabab1ead06ebc/family1 is initiating major compaction (all files) 2023-02-08 23:55:52,109 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:52,110 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:52,110 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of 9d98bce82dc93804dedabab1ead06ebc/family1 in ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:52,111 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/a00d3474a0554ab6968f17669908b78a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/7f9bd460450f40c3b55e63b309ff3eba, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/8ddb16124e6b49f99bfdeb1a415b184e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/4087fff1c9dd4948908faee98d380124, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/fa86b37d11de4e6194560afffb5a23de] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp, totalSize=26.6 K 2023-02-08 23:55:52,111 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting a00d3474a0554ab6968f17669908b78a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900540595 2023-02-08 23:55:52,111 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 7f9bd460450f40c3b55e63b309ff3eba, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900542400 2023-02-08 23:55:52,112 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 8ddb16124e6b49f99bfdeb1a415b184e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900544212 2023-02-08 23:55:52,112 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 4087fff1c9dd4948908faee98d380124, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900545984 2023-02-08 23:55:52,113 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting fa86b37d11de4e6194560afffb5a23de, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:55:52,121 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): 9d98bce82dc93804dedabab1ead06ebc#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:52,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/e656241afc5e414cb19861b9674b6db2, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/270ed97bb77f422fbaf9d8d18f5187ed, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/003d63aa5bfc41d794c4dd002f343159, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/16e018eeea794fa4aedd9f090ad09075, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/bb6034ca556048aebe308d070f5a5675] to archive 2023-02-08 23:55:52,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:52,126 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/e656241afc5e414cb19861b9674b6db2 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/e656241afc5e414cb19861b9674b6db2 2023-02-08 23:55:52,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/270ed97bb77f422fbaf9d8d18f5187ed to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/270ed97bb77f422fbaf9d8d18f5187ed 2023-02-08 23:55:52,129 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302084c6d3d13a5154cd2af3178dc52e0f889_del 2023-02-08 23:55:52,129 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302084c6d3d13a5154cd2af3178dc52e0f889_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302084c6d3d13a5154cd2af3178dc52e0f889_del 2023-02-08 23:55:52,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/003d63aa5bfc41d794c4dd002f343159 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/003d63aa5bfc41d794c4dd002f343159 2023-02-08 23:55:52,131 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/16e018eeea794fa4aedd9f090ad09075 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/16e018eeea794fa4aedd9f090ad09075 2023-02-08 23:55:52,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/bb6034ca556048aebe308d070f5a5675 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family2/bb6034ca556048aebe308d070f5a5675 2023-02-08 23:55:52,946 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8934fe4a148444b09caa14fae5babc32 2023-02-08 23:55:52,947 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family1/8934fe4a148444b09caa14fae5babc32 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/8934fe4a148444b09caa14fae5babc32 2023-02-08 23:55:52,951 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8934fe4a148444b09caa14fae5babc32 2023-02-08 23:55:52,952 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9d98bce82dc93804dedabab1ead06ebc/family1 of 9d98bce82dc93804dedabab1ead06ebc into 8934fe4a148444b09caa14fae5babc32(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:52,952 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for 9d98bce82dc93804dedabab1ead06ebc: 2023-02-08 23:55:52,952 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:52,952 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): 9d98bce82dc93804dedabab1ead06ebc/family2 is initiating major compaction (all files) 2023-02-08 23:55:52,952 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:52,952 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:52,953 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of 9d98bce82dc93804dedabab1ead06ebc/family2 in ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:55:52,953 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/630e65eb95704d27a9433b83eee4b79c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/053635334e9b41d1875e1fe3b0b547ff, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/85a66618c02546ec8241d11dbe24a284, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/8bb63c6dde544f8d8ca36ef866fcce14, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/5d2fdb45255849748b66c2604e916be2] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp, totalSize=25.3 K 2023-02-08 23:55:52,953 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 630e65eb95704d27a9433b83eee4b79c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900540595 2023-02-08 23:55:52,954 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 053635334e9b41d1875e1fe3b0b547ff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900542400 2023-02-08 23:55:52,954 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 85a66618c02546ec8241d11dbe24a284, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900544212 2023-02-08 23:55:52,955 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 8bb63c6dde544f8d8ca36ef866fcce14, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900545984 2023-02-08 23:55:52,960 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 5d2fdb45255849748b66c2604e916be2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:55:52,968 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): 9d98bce82dc93804dedabab1ead06ebc#family2#compaction#152 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:53,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/a00d3474a0554ab6968f17669908b78a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/7f9bd460450f40c3b55e63b309ff3eba, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/8ddb16124e6b49f99bfdeb1a415b184e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/4087fff1c9dd4948908faee98d380124, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/fa86b37d11de4e6194560afffb5a23de] to archive 2023-02-08 23:55:53,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:53,031 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/a00d3474a0554ab6968f17669908b78a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/a00d3474a0554ab6968f17669908b78a 2023-02-08 23:55:53,033 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/7f9bd460450f40c3b55e63b309ff3eba to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/7f9bd460450f40c3b55e63b309ff3eba 2023-02-08 23:55:53,034 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/8ddb16124e6b49f99bfdeb1a415b184e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/8ddb16124e6b49f99bfdeb1a415b184e 2023-02-08 23:55:53,035 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/4087fff1c9dd4948908faee98d380124 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/4087fff1c9dd4948908faee98d380124 2023-02-08 23:55:53,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/fa86b37d11de4e6194560afffb5a23de to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/fa86b37d11de4e6194560afffb5a23de 2023-02-08 23:55:53,377 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302086a35f7308f734088919d862d5a681cf6_del 2023-02-08 23:55:53,377 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302086a35f7308f734088919d862d5a681cf6_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302086a35f7308f734088919d862d5a681cf6_del 2023-02-08 23:55:53,795 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7b2d019c3004669a576c0794592e047 2023-02-08 23:55:53,795 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/.tmp/family2/e7b2d019c3004669a576c0794592e047 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/e7b2d019c3004669a576c0794592e047 2023-02-08 23:55:53,799 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7b2d019c3004669a576c0794592e047 2023-02-08 23:55:53,800 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9d98bce82dc93804dedabab1ead06ebc/family2 of 9d98bce82dc93804dedabab1ead06ebc into e7b2d019c3004669a576c0794592e047(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:53,800 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for 9d98bce82dc93804dedabab1ead06ebc: 2023-02-08 23:55:53,800 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:53,800 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): 9246c8eb0029ec8f06f1acd496f0cde7/family1 is initiating major compaction (all files) 2023-02-08 23:55:53,801 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:53,801 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:53,801 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of 9246c8eb0029ec8f06f1acd496f0cde7/family1 in ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:53,801 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/45f8c1677678405c9832becbdccd6a4e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/0927e87ec7f44df9aa6c9eaaad2a1e60, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/667901d58f15419e92c54106c66bfc09, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/47f7697e55784f9290cd96b0f99dac54, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/ff704c6b0f28411cb605e2b9e6e01467] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp, totalSize=26.6 K 2023-02-08 23:55:53,801 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 45f8c1677678405c9832becbdccd6a4e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900540596 2023-02-08 23:55:53,802 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 0927e87ec7f44df9aa6c9eaaad2a1e60, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900542400 2023-02-08 23:55:53,802 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 667901d58f15419e92c54106c66bfc09, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900544212 2023-02-08 23:55:53,803 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 47f7697e55784f9290cd96b0f99dac54, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900545984 2023-02-08 23:55:53,803 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting ff704c6b0f28411cb605e2b9e6e01467, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:55:53,813 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): 9246c8eb0029ec8f06f1acd496f0cde7#family1#compaction#153 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:53,825 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/630e65eb95704d27a9433b83eee4b79c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/053635334e9b41d1875e1fe3b0b547ff, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/85a66618c02546ec8241d11dbe24a284, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/8bb63c6dde544f8d8ca36ef866fcce14, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/5d2fdb45255849748b66c2604e916be2] to archive 2023-02-08 23:55:53,825 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:53,827 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/630e65eb95704d27a9433b83eee4b79c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/630e65eb95704d27a9433b83eee4b79c 2023-02-08 23:55:53,828 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/053635334e9b41d1875e1fe3b0b547ff to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/053635334e9b41d1875e1fe3b0b547ff 2023-02-08 23:55:53,829 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/85a66618c02546ec8241d11dbe24a284 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/85a66618c02546ec8241d11dbe24a284 2023-02-08 23:55:53,830 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/8bb63c6dde544f8d8ca36ef866fcce14 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/8bb63c6dde544f8d8ca36ef866fcce14 2023-02-08 23:55:53,831 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/5d2fdb45255849748b66c2604e916be2 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family2/5d2fdb45255849748b66c2604e916be2 2023-02-08 23:55:54,221 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302081da082b7bc244be3a2fce5aacba726ce_del 2023-02-08 23:55:54,222 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302081da082b7bc244be3a2fce5aacba726ce_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302081da082b7bc244be3a2fce5aacba726ce_del 2023-02-08 23:55:54,637 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06df4fc0dbf243d2aee0c903d640d056 2023-02-08 23:55:54,638 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family1/06df4fc0dbf243d2aee0c903d640d056 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/06df4fc0dbf243d2aee0c903d640d056 2023-02-08 23:55:54,642 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06df4fc0dbf243d2aee0c903d640d056 2023-02-08 23:55:54,642 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9246c8eb0029ec8f06f1acd496f0cde7/family1 of 9246c8eb0029ec8f06f1acd496f0cde7 into 06df4fc0dbf243d2aee0c903d640d056(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:54,643 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for 9246c8eb0029ec8f06f1acd496f0cde7: 2023-02-08 23:55:54,643 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:55:54,643 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): 9246c8eb0029ec8f06f1acd496f0cde7/family2 is initiating major compaction (all files) 2023-02-08 23:55:54,643 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:55:54,643 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:55:54,643 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of 9246c8eb0029ec8f06f1acd496f0cde7/family2 in ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:55:54,643 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4b63be95dda64b7fbab51ee6ab2ca55e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/9b55e41a4ac9400aa7db075985870b2f, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4117d092636743c68da983412024d64e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/bdfaacff8a0d459ca0e5d57f13a693a1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/536ac926736f422ea3968d6b7f62d8f5] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp, totalSize=25.3 K 2023-02-08 23:55:54,644 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 4b63be95dda64b7fbab51ee6ab2ca55e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900540596 2023-02-08 23:55:54,644 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 9b55e41a4ac9400aa7db075985870b2f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900542400 2023-02-08 23:55:54,644 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 4117d092636743c68da983412024d64e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900544212 2023-02-08 23:55:54,644 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting bdfaacff8a0d459ca0e5d57f13a693a1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900545984 2023-02-08 23:55:54,645 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 536ac926736f422ea3968d6b7f62d8f5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:55:54,658 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): 9246c8eb0029ec8f06f1acd496f0cde7#family2#compaction#154 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:55:54,665 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230208eb074fd0d2f04291ac1e37b362d61ad8_del 2023-02-08 23:55:54,665 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208eb074fd0d2f04291ac1e37b362d61ad8_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230208eb074fd0d2f04291ac1e37b362d61ad8_del 2023-02-08 23:55:54,723 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/45f8c1677678405c9832becbdccd6a4e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/0927e87ec7f44df9aa6c9eaaad2a1e60, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/667901d58f15419e92c54106c66bfc09, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/47f7697e55784f9290cd96b0f99dac54, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/ff704c6b0f28411cb605e2b9e6e01467] to archive 2023-02-08 23:55:54,724 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:54,726 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/45f8c1677678405c9832becbdccd6a4e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/45f8c1677678405c9832becbdccd6a4e 2023-02-08 23:55:54,727 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/0927e87ec7f44df9aa6c9eaaad2a1e60 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/0927e87ec7f44df9aa6c9eaaad2a1e60 2023-02-08 23:55:54,728 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/667901d58f15419e92c54106c66bfc09 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/667901d58f15419e92c54106c66bfc09 2023-02-08 23:55:54,729 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/47f7697e55784f9290cd96b0f99dac54 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/47f7697e55784f9290cd96b0f99dac54 2023-02-08 23:55:54,730 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/ff704c6b0f28411cb605e2b9e6e01467 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/ff704c6b0f28411cb605e2b9e6e01467 2023-02-08 23:55:55,486 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21a964c2723b45ff94d83349a1df8417 2023-02-08 23:55:55,487 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/.tmp/family2/21a964c2723b45ff94d83349a1df8417 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/21a964c2723b45ff94d83349a1df8417 2023-02-08 23:55:55,491 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21a964c2723b45ff94d83349a1df8417 2023-02-08 23:55:55,492 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9246c8eb0029ec8f06f1acd496f0cde7/family2 of 9246c8eb0029ec8f06f1acd496f0cde7 into 21a964c2723b45ff94d83349a1df8417(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:55:55,492 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for 9246c8eb0029ec8f06f1acd496f0cde7: 2023-02-08 23:55:55,496 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:55,499 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:55,502 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:55,503 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:55,508 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 23:55:55,520 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-08 23:55:55,522 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302084c6d3d13a5154cd2af3178dc52e0f889_del 2023-02-08 23:55:55,524 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4b63be95dda64b7fbab51ee6ab2ca55e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/9b55e41a4ac9400aa7db075985870b2f, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4117d092636743c68da983412024d64e, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/bdfaacff8a0d459ca0e5d57f13a693a1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/536ac926736f422ea3968d6b7f62d8f5] to archive 2023-02-08 23:55:55,524 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:55,525 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302081da082b7bc244be3a2fce5aacba726ce_del 2023-02-08 23:55:55,526 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4b63be95dda64b7fbab51ee6ab2ca55e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4b63be95dda64b7fbab51ee6ab2ca55e 2023-02-08 23:55:55,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/9b55e41a4ac9400aa7db075985870b2f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/9b55e41a4ac9400aa7db075985870b2f 2023-02-08 23:55:55,528 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4117d092636743c68da983412024d64e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/4117d092636743c68da983412024d64e 2023-02-08 23:55:55,529 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023020887314bb284844b9d8889a36e5d8afffb_del 2023-02-08 23:55:55,529 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-08 23:55:55,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/bdfaacff8a0d459ca0e5d57f13a693a1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/bdfaacff8a0d459ca0e5d57f13a693a1 2023-02-08 23:55:55,529 DEBUG [Listener at localhost/41521] zookeeper.ReadOnlyZKClient(139): Connect 0x2c3f2136 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:55:55,532 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/536ac926736f422ea3968d6b7f62d8f5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family2/536ac926736f422ea3968d6b7f62d8f5 2023-02-08 23:55:55,534 DEBUG [Listener at localhost/41521] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3215ecc8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:55:55,535 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 23:55:55,535 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 23:55:55,535 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 23:55:55,976 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208f19c624fcf2140328ad1df1a3d13bb83 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230208f19c624fcf2140328ad1df1a3d13bb83 2023-02-08 23:55:55,977 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 23:55:55,979 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208cb576a5dd3914233a383cf9ac0bdcdfe to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230208cb576a5dd3914233a383cf9ac0bdcdfe 2023-02-08 23:55:55,980 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 23:55:55,980 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208cdb1b5e7fc884ac9ab6bd7da4847c703 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230208cdb1b5e7fc884ac9ab6bd7da4847c703 2023-02-08 23:55:55,984 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 23:55:55,997 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:55:56,000 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47858, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:55:56,010 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:56,010 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:56,011 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:55:56,015 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40768, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:55:56,023 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:56,023 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:56,023 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:56,023 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:56,051 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208/family1/8aa1fe6fb07b4bac925598bd3e957b1b first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-08 23:55:56,052 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208/family1/8aa1fe6fb07b4bac925598bd3e957b1b}] 2023-02-08 23:55:56,052 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/family1/dcef28abef1a4354af54d75f8b2e795a first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-08 23:55:56,056 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/family1/dcef28abef1a4354af54d75f8b2e795a}] 2023-02-08 23:55:56,060 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208/family1/bd7cabcf1ffd49c6a0482eb4657c2ad2 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-08 23:55:56,065 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208/family1/bd7cabcf1ffd49c6a0482eb4657c2ad2}] 2023-02-08 23:55:56,068 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208/family1/8aa1fe6fb07b4bac925598bd3e957b1b for inclusion in 539644099f8843cf0be59ce746f2f9e3/family1 2023-02-08 23:55:56,068 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/family1/dcef28abef1a4354af54d75f8b2e795a for inclusion in 9d98bce82dc93804dedabab1ead06ebc/family1 2023-02-08 23:55:56,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208/family1/bd7cabcf1ffd49c6a0482eb4657c2ad2 for inclusion in 9246c8eb0029ec8f06f1acd496f0cde7/family1 2023-02-08 23:55:56,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-08 23:55:56,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 23:55:56,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 539644099f8843cf0be59ce746f2f9e3: 2023-02-08 23:55:56,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-08 23:55:56,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last=1 2023-02-08 23:55:56,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 9d98bce82dc93804dedabab1ead06ebc: 2023-02-08 23:55:56,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208/family1/8aa1fe6fb07b4bac925598bd3e957b1b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__858v1e5vecams60a02o7hhqmeus2172n3gq6ejcdssnmj7jna6ikn25elsp16bji/family1/8aa1fe6fb07b4bac925598bd3e957b1b 2023-02-08 23:55:56,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/family1/dcef28abef1a4354af54d75f8b2e795a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__7rva3ttj6m7r12tgvhq6qjcda72g2gv0usm5hil6vuq22t0eebke3a267caimqj7/family1/dcef28abef1a4354af54d75f8b2e795a 2023-02-08 23:55:56,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-08 23:55:56,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-08 23:55:56,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 9246c8eb0029ec8f06f1acd496f0cde7: 2023-02-08 23:55:56,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208/family1/bd7cabcf1ffd49c6a0482eb4657c2ad2 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__moe3ou67k4tuveu11ehu2ti01p9cdtf45f1ej94u5n18pomnft6q6alc5vgjqmof/family1/bd7cabcf1ffd49c6a0482eb4657c2ad2 2023-02-08 23:55:56,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__7rva3ttj6m7r12tgvhq6qjcda72g2gv0usm5hil6vuq22t0eebke3a267caimqj7/family1/dcef28abef1a4354af54d75f8b2e795a as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/0618c256fba84d739739ab2d969a572c_SeqId_28_ 2023-02-08 23:55:56,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__858v1e5vecams60a02o7hhqmeus2172n3gq6ejcdssnmj7jna6ikn25elsp16bji/family1/8aa1fe6fb07b4bac925598bd3e957b1b as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/044d037ea6e64c19aa4bc77a60059d40_SeqId_28_ 2023-02-08 23:55:56,085 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__7rva3ttj6m7r12tgvhq6qjcda72g2gv0usm5hil6vuq22t0eebke3a267caimqj7/family1/dcef28abef1a4354af54d75f8b2e795a into 9d98bce82dc93804dedabab1ead06ebc/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/0618c256fba84d739739ab2d969a572c_SeqId_28_ - updating store file list. 2023-02-08 23:55:56,085 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__858v1e5vecams60a02o7hhqmeus2172n3gq6ejcdssnmj7jna6ikn25elsp16bji/family1/8aa1fe6fb07b4bac925598bd3e957b1b into 539644099f8843cf0be59ce746f2f9e3/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/044d037ea6e64c19aa4bc77a60059d40_SeqId_28_ - updating store file list. 2023-02-08 23:55:56,089 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/0618c256fba84d739739ab2d969a572c_SeqId_28_ into 9d98bce82dc93804dedabab1ead06ebc/family1 2023-02-08 23:55:56,089 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__7rva3ttj6m7r12tgvhq6qjcda72g2gv0usm5hil6vuq22t0eebke3a267caimqj7/family1/dcef28abef1a4354af54d75f8b2e795a into 9d98bce82dc93804dedabab1ead06ebc/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/family1/0618c256fba84d739739ab2d969a572c_SeqId_28_) 2023-02-08 23:55:56,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__moe3ou67k4tuveu11ehu2ti01p9cdtf45f1ej94u5n18pomnft6q6alc5vgjqmof/family1/bd7cabcf1ffd49c6a0482eb4657c2ad2 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/d0d1883589294624bd88c9d4c9d46313_SeqId_28_ 2023-02-08 23:55:56,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__7rva3ttj6m7r12tgvhq6qjcda72g2gv0usm5hil6vuq22t0eebke3a267caimqj7/family1/dcef28abef1a4354af54d75f8b2e795a 2023-02-08 23:55:56,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__moe3ou67k4tuveu11ehu2ti01p9cdtf45f1ej94u5n18pomnft6q6alc5vgjqmof/family1/bd7cabcf1ffd49c6a0482eb4657c2ad2 into 9246c8eb0029ec8f06f1acd496f0cde7/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/d0d1883589294624bd88c9d4c9d46313_SeqId_28_ - updating store file list. 2023-02-08 23:55:56,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/044d037ea6e64c19aa4bc77a60059d40_SeqId_28_ into 539644099f8843cf0be59ce746f2f9e3/family1 2023-02-08 23:55:56,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__858v1e5vecams60a02o7hhqmeus2172n3gq6ejcdssnmj7jna6ikn25elsp16bji/family1/8aa1fe6fb07b4bac925598bd3e957b1b into 539644099f8843cf0be59ce746f2f9e3/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/family1/044d037ea6e64c19aa4bc77a60059d40_SeqId_28_) 2023-02-08 23:55:56,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__858v1e5vecams60a02o7hhqmeus2172n3gq6ejcdssnmj7jna6ikn25elsp16bji/family1/8aa1fe6fb07b4bac925598bd3e957b1b 2023-02-08 23:55:56,094 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 23:55:56,094 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 23:55:56,095 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:56,095 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:56,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/d0d1883589294624bd88c9d4c9d46313_SeqId_28_ into 9246c8eb0029ec8f06f1acd496f0cde7/family1 2023-02-08 23:55:56,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__moe3ou67k4tuveu11ehu2ti01p9cdtf45f1ej94u5n18pomnft6q6alc5vgjqmof/family1/bd7cabcf1ffd49c6a0482eb4657c2ad2 into 9246c8eb0029ec8f06f1acd496f0cde7/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/family1/d0d1883589294624bd88c9d4c9d46313_SeqId_28_) 2023-02-08 23:55:56,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__ns_testMinorCompaction__moe3ou67k4tuveu11ehu2ti01p9cdtf45f1ej94u5n18pomnft6q6alc5vgjqmof/family1/bd7cabcf1ffd49c6a0482eb4657c2ad2 2023-02-08 23:55:56,097 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230208f61416886a5c481b968b545444ff77b9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230208f61416886a5c481b968b545444ff77b9 2023-02-08 23:55:56,097 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-08 23:55:56,097 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 23:55:56,098 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 23:55:56,098 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 23:55:56,098 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302080315ab895e664b958006717392466084 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302080315ab895e664b958006717392466084 2023-02-08 23:55:56,098 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-08 23:55:56,098 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 23:55:56,098 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:55:56,099 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 23:55:56,100 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230208b0c4ef14239f4fa69ca891e175a58484 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230208b0c4ef14239f4fa69ca891e175a58484 2023-02-08 23:55:56,100 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-08 23:55:56,100 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208' 2023-02-08 23:55:56,101 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 23:55:56,101 INFO [Listener at localhost/41521] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-08 23:55:56,103 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:56,105 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:56,108 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:55:56,109 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:55:56,130 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=252 (was 282), OpenFileDescriptor=645 (was 749), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=100 (was 118), ProcessCount=168 (was 168), AvailableMemoryMB=3385 (was 3070) - AvailableMemoryMB LEAK? - 2023-02-08 23:55:56,146 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=252, OpenFileDescriptor=645, MaxFileDescriptor=60000, SystemLoadAverage=100, ProcessCount=167, AvailableMemoryMB=3384 2023-02-08 23:55:56,147 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:56,147 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:56,148 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 23:55:56,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:56,151 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:55:56,151 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-08 23:55:56,151 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:56,152 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:56,152 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:55:56,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 23:55:56,153 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:55:56,154 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4 empty. 2023-02-08 23:55:56,154 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:55:56,154 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-08 23:55:56,253 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:56,253 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:56,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 23:55:56,454 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:56,454 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:56,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 23:55:56,571 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-08 23:55:56,572 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 56b1dbb83cb5199116335c5260eadbb4, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:55:56,756 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:56,756 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:56,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 23:55:56,977 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:56,977 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 56b1dbb83cb5199116335c5260eadbb4, disabling compactions & flushes 2023-02-08 23:55:56,978 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:56,978 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:56,978 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. after waiting 0 ms 2023-02-08 23:55:56,978 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:56,978 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:56,978 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 56b1dbb83cb5199116335c5260eadbb4: Waiting for close lock at 1675900556977Disabling compacts and flushes for region at 1675900556977Disabling writes for close at 1675900556978 (+1 ms)Writing region close event to WAL at 1675900556978Closed at 1675900556978 2023-02-08 23:55:56,980 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:55:56,981 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900556981"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900556981"}]},"ts":"1675900556981"} 2023-02-08 23:55:56,982 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 23:55:56,983 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:55:56,983 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900556983"}]},"ts":"1675900556983"} 2023-02-08 23:55:56,983 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-08 23:55:56,987 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=56b1dbb83cb5199116335c5260eadbb4, ASSIGN}] 2023-02-08 23:55:56,989 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=56b1dbb83cb5199116335c5260eadbb4, ASSIGN 2023-02-08 23:55:56,989 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=56b1dbb83cb5199116335c5260eadbb4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:55:57,140 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=56b1dbb83cb5199116335c5260eadbb4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:57,141 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900557140"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900557140"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900557140"}]},"ts":"1675900557140"} 2023-02-08 23:55:57,142 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 56b1dbb83cb5199116335c5260eadbb4, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:55:57,257 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:57,257 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:57,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 23:55:57,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:57,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 56b1dbb83cb5199116335c5260eadbb4, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4.', STARTKEY => '', ENDKEY => ''} 2023-02-08 23:55:57,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:55:57,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:55:57,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:55:57,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:55:57,298 INFO [StoreOpener-56b1dbb83cb5199116335c5260eadbb4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:55:57,299 DEBUG [StoreOpener-56b1dbb83cb5199116335c5260eadbb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1 2023-02-08 23:55:57,299 DEBUG [StoreOpener-56b1dbb83cb5199116335c5260eadbb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1 2023-02-08 23:55:57,299 INFO [StoreOpener-56b1dbb83cb5199116335c5260eadbb4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 56b1dbb83cb5199116335c5260eadbb4 columnFamilyName family1 2023-02-08 23:55:57,301 INFO [StoreOpener-56b1dbb83cb5199116335c5260eadbb4-1] regionserver.HStore(310): Store=56b1dbb83cb5199116335c5260eadbb4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:55:57,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:55:57,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:55:57,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:55:57,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:55:57,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 56b1dbb83cb5199116335c5260eadbb4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72915254, jitterRate=0.08652195334434509}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 23:55:57,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 56b1dbb83cb5199116335c5260eadbb4: Running coprocessor pre-open hook at 1675900557297Writing region info on filesystem at 1675900557297Initializing all the Stores at 1675900557297Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675900557298 (+1 ms)Cleaning up temporary data from old regions at 1675900557304 (+6 ms)Cleaning up detritus from prior splits at 1675900557305 (+1 ms)Running coprocessor post-open hooks at 1675900557308 (+3 ms)Region opened successfully at 1675900557308 2023-02-08 23:55:57,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4., pid=114, masterSystemTime=1675900557293 2023-02-08 23:55:57,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:57,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:57,310 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=56b1dbb83cb5199116335c5260eadbb4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:57,311 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675900557310"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900557310"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900557310"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900557310"}]},"ts":"1675900557310"} 2023-02-08 23:55:57,313 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-08 23:55:57,313 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 56b1dbb83cb5199116335c5260eadbb4, server=jenkins-hbase4.apache.org,42703,1675900414768 in 170 msec 2023-02-08 23:55:57,314 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-08 23:55:57,314 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=56b1dbb83cb5199116335c5260eadbb4, ASSIGN in 326 msec 2023-02-08 23:55:57,315 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:55:57,315 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900557315"}]},"ts":"1675900557315"} 2023-02-08 23:55:57,316 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-08 23:55:57,318 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:55:57,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1690 sec 2023-02-08 23:55:58,258 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:58,259 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:58,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 23:55:58,260 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-08 23:55:58,276 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:58,276 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:58,277 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:58,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:58,279 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:55:58,279 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:58,279 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:55:58,279 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:55:58,279 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:58,279 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:58,281 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:58,281 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:58,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:58,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:58,281 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:58,281 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:58,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:58,281 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:58,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:58,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:58,282 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:58,289 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:58,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:55:58,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:58,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:55:58,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:58,289 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:58,289 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:58,289 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:55:58,290 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 23:55:58,711 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113002ec42eb488747e38c16593e8450a192 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113002ec42eb488747e38c16593e8450a192 2023-02-08 23:55:59,114 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/74c56c3666464462a505e6cfe365ee7d 2023-02-08 23:55:59,119 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/74c56c3666464462a505e6cfe365ee7d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/74c56c3666464462a505e6cfe365ee7d 2023-02-08 23:55:59,123 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/74c56c3666464462a505e6cfe365ee7d, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-08 23:55:59,123 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 833ms, sequenceid=5, compaction requested=false 2023-02-08 23:55:59,124 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:55:59,124 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:55:59,124 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:55:59,124 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:55:59,127 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,127 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:59,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:59,127 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,127 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:55:59,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:59,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:55:59,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25c195dd[Count = 0] remaining members to acquire global barrier 2023-02-08 23:55:59,128 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:55:59,128 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,129 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:55:59,129 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,130 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:55:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:55:59,130 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:55:59,131 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:55:59,131 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,131 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:59,131 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:59,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:55:59,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:55:59,133 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:55:59,133 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:55:59,133 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:55:59,133 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,133 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:55:59,134 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,134 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:55:59,135 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:59,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,135 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:55:59,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,136 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:55:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,139 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,139 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:59,139 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:59,139 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:55:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:59,139 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,140 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:55:59,140 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:55:59,140 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:55:59,140 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:55:59,140 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:55:59,140 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,140 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:55:59,140 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:55:59,140 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,240 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:55:59,240 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:59,240 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:59,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:55:59,244 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:55:59,244 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:55:59,245 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:55:59,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:55:59,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,247 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:55:59,247 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:59,247 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:55:59,247 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:55:59,247 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,247 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,248 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,248 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:55:59,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:55:59,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:55:59,248 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:55:59,248 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:55:59,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,249 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:55:59,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:55:59,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:55:59,249 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:55:59,249 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:55:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:55:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:55:59,250 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:55:59,250 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:55:59,250 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:55:59,250 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 23:55:59,665 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128fc4e102addcb406bb8f94bc38696c696 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128fc4e102addcb406bb8f94bc38696c696 2023-02-08 23:56:00,070 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/5f8493689de541b5899e30053d800b5a 2023-02-08 23:56:00,074 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/5f8493689de541b5899e30053d800b5a as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/5f8493689de541b5899e30053d800b5a 2023-02-08 23:56:00,077 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/5f8493689de541b5899e30053d800b5a, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 23:56:00,078 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 828ms, sequenceid=9, compaction requested=false 2023-02-08 23:56:00,078 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:00,078 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:56:00,080 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,080 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:00,080 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,080 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:00,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:00,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:56:00,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11bdffc1[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:00,081 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:56:00,081 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,082 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,082 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:56:00,082 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:56:00,082 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,083 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:00,083 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:00,085 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,085 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:56:00,085 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:00,085 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:00,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:56:00,087 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:56:00,087 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:00,087 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:00,087 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,087 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:00,088 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,088 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:00,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:00,088 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:00,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,088 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:00,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,089 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:00,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,091 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:00,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:00,092 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:00,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:00,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:00,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:00,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:56:00,091 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,092 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,092 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:00,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:00,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:00,092 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,092 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:00,192 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:00,193 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:00,193 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:00,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:00,196 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:00,197 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:00,197 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:00,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:00,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,199 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:00,199 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:00,199 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:56:00,199 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:56:00,199 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,200 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,205 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,205 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:00,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:00,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:00,205 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,205 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:00,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,206 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:00,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,206 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,206 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:56:00,206 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,206 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:56:00,206 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:00,206 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:56:00,207 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:56:00,207 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:00,207 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:00,207 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:56:00,207 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:56:00,221 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201270600c2096748b59ce163a7648094be to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201270600c2096748b59ce163a7648094be 2023-02-08 23:56:00,625 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/f9371b84ab93440eae41a54d7f6ac91d 2023-02-08 23:56:00,629 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/f9371b84ab93440eae41a54d7f6ac91d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/f9371b84ab93440eae41a54d7f6ac91d 2023-02-08 23:56:00,632 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/f9371b84ab93440eae41a54d7f6ac91d, entries=1, sequenceid=13, filesize=4.9 K 2023-02-08 23:56:00,632 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 425ms, sequenceid=13, compaction requested=true 2023-02-08 23:56:00,633 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:00,633 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:00,633 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:56:00,633 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:56:00,633 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,633 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:56:00,633 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:56:00,634 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,634 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:00,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:00,634 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,635 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:00,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:56:00,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64be7be4[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:00,636 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:56:00,636 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,637 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,637 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:56:00,637 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:56:00,637 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:00,637 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,638 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:00,643 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:56:00,643 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:00,643 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,643 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:56:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:00,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:00,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:00,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:56:00,645 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:56:00,645 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:00,645 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:00,645 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,645 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:00,647 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,647 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:00,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,647 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:00,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:00,647 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:00,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:00,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,648 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:00,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,650 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,650 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:00,650 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:00,650 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:00,650 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:00,650 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:00,650 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:00,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:56:00,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:00,650 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,651 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,651 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,651 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,651 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,651 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:00,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:00,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:00,651 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:00,751 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:00,751 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:00,751 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:00,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:00,756 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:00,756 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:00,756 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:00,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:00,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,758 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:00,758 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:00,758 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:56:00,758 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:56:00,758 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,758 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,759 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:00,759 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:00,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:00,760 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:00,760 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:00,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:00,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,761 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:56:00,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:56:00,761 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:00,761 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:00,761 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:56:00,761 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:56:01,172 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205147dd31ea6aa41bc8ec0ff157399283d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205147dd31ea6aa41bc8ec0ff157399283d 2023-02-08 23:56:01,577 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/65f8adc54869482c858f3425761a5bc5 2023-02-08 23:56:01,581 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/65f8adc54869482c858f3425761a5bc5 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65f8adc54869482c858f3425761a5bc5 2023-02-08 23:56:01,584 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65f8adc54869482c858f3425761a5bc5, entries=1, sequenceid=17, filesize=4.9 K 2023-02-08 23:56:01,585 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 824ms, sequenceid=17, compaction requested=true 2023-02-08 23:56:01,585 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:01,585 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:56:01,588 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,588 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:01,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:01,589 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,589 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:01,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:01,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:01,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:01,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:56:01,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d126646[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:01,590 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:56:01,590 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,591 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,591 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:56:01,591 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:56:01,591 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:01,591 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,591 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:01,592 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:56:01,592 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:01,592 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,592 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:56:01,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:01,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:01,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:01,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:56:01,594 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:56:01,594 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:01,594 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:01,594 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,594 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:01,595 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,595 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:01,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,595 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:01,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:01,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:01,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:01,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:01,596 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:01,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,597 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:01,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,600 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,600 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:01,600 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:01,600 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:01,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:01,600 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:01,600 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:01,600 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:01,600 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,601 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,601 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:01,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:01,601 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,601 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:01,601 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:01,601 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,701 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:01,701 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:01,701 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:01,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:01,705 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:01,705 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:01,706 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:01,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:01,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,707 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:01,707 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:01,707 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:56:01,707 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:56:01,708 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,708 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,709 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:01,709 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:01,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:01,709 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:01,709 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:01,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:01,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:56:01,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:01,712 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:56:01,712 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:01,712 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:56:01,712 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:56:01,712 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:01,712 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:01,712 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:56:01,712 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:56:01,725 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d39e097da81645db8479344ccfe2cef8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228d39e097da81645db8479344ccfe2cef8 2023-02-08 23:56:02,130 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/40a1ae784aa042f994748fc316a98782 2023-02-08 23:56:02,135 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/40a1ae784aa042f994748fc316a98782 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/40a1ae784aa042f994748fc316a98782 2023-02-08 23:56:02,139 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/40a1ae784aa042f994748fc316a98782, entries=1, sequenceid=21, filesize=4.9 K 2023-02-08 23:56:02,139 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 427ms, sequenceid=21, compaction requested=true 2023-02-08 23:56:02,139 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:02,140 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:02,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:56:02,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:56:02,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:56:02,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:56:02,141 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,141 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:02,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:02,142 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:02,142 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:02,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:02,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:02,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:56:02,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24d46544[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:02,143 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:56:02,143 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,144 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,144 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:56:02,144 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,144 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:02,144 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:56:02,144 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:02,147 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,147 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:56:02,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:02,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:02,147 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:02,148 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:56:02,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:02,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:02,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:02,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:02,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:56:02,149 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:56:02,150 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:02,150 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:02,150 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,150 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:02,151 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,151 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,151 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:02,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:02,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:02,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:02,152 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,153 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:02,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:02,157 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,157 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:02,157 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:02,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:02,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:02,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:02,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:56:02,157 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:02,258 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:02,258 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:02,258 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:02,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:02,262 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:02,262 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:02,263 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:02,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:02,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,264 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:02,264 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:02,264 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:56:02,264 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:56:02,265 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,265 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,266 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:02,266 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:02,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:02,266 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:02,266 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:02,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:02,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,267 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,267 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:56:02,267 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:02,267 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:56:02,267 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:02,267 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:56:02,267 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:56:02,267 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:02,268 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:02,268 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:56:02,268 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:56:02,679 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d6f1d95ce88043d4819c73d11e907121 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231d6f1d95ce88043d4819c73d11e907121 2023-02-08 23:56:02,833 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:56:03,084 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/b9a9d786dc5f42548f11ce4a2421a1c1 2023-02-08 23:56:03,088 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/b9a9d786dc5f42548f11ce4a2421a1c1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/b9a9d786dc5f42548f11ce4a2421a1c1 2023-02-08 23:56:03,091 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/b9a9d786dc5f42548f11ce4a2421a1c1, entries=1, sequenceid=25, filesize=4.9 K 2023-02-08 23:56:03,092 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 824ms, sequenceid=25, compaction requested=true 2023-02-08 23:56:03,092 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:03,092 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:03,092 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:56:03,092 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:56:03,092 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,092 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:56:03,092 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:56:03,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,094 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:03,094 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,094 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ae9d5e8[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:03,095 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:56:03,095 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,097 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,098 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:56:03,098 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:56:03,098 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:03,098 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,098 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:03,099 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,099 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:56:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:03,099 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:03,099 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:56:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:03,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:03,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:56:03,101 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:56:03,101 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:03,101 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:03,101 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,101 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:03,102 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,102 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:03,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,102 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:03,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:03,103 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:03,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,104 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:03,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:03,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:03,106 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:03,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:03,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:03,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:03,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:03,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:03,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:03,107 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:03,107 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:03,207 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:03,207 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:03,207 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:03,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:03,211 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:03,211 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:03,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:03,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:03,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,213 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:03,213 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:03,213 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:56:03,213 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:56:03,213 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,213 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,215 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:03,215 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:03,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:03,215 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,215 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:03,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:03,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,216 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,216 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:56:03,216 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:56:03,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:03,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:56:03,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:56:03,216 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:03,216 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:03,216 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:56:03,217 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:56:03,226 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c2dfb292283b453391d2cf99f6ffe3f8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101c2dfb292283b453391d2cf99f6ffe3f8 2023-02-08 23:56:03,230 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/398ace704e6b4f1d887932275813fc28 2023-02-08 23:56:03,233 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/398ace704e6b4f1d887932275813fc28 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/398ace704e6b4f1d887932275813fc28 2023-02-08 23:56:03,236 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/398ace704e6b4f1d887932275813fc28, entries=1, sequenceid=29, filesize=4.9 K 2023-02-08 23:56:03,237 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 20ms, sequenceid=29, compaction requested=true 2023-02-08 23:56:03,237 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:03,237 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:03,237 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:56:03,237 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:56:03,237 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,237 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:56:03,237 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:56:03,238 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,239 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:03,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:03,239 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,239 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:03,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:03,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:03,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:03,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:56:03,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d1fe40a[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:03,240 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:56:03,240 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,241 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,241 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:56:03,242 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:56:03,242 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,242 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:03,242 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:03,243 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:56:03,243 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,243 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:03,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,243 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:56:03,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:03,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:03,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:03,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:03,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:03,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:03,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:56:03,245 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:56:03,245 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:03,245 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:03,245 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,245 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:03,247 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,247 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:03,248 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,248 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:03,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,249 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:03,251 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:03,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:03,251 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:03,251 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:03,251 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:03,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:03,252 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:03,251 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,352 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:03,352 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:03,352 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:03,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:03,356 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:03,356 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:03,356 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:03,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:03,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,358 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:03,358 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:03,358 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:56:03,358 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:56:03,358 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,358 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,359 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,359 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:03,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:03,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:03,360 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:03,360 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:03,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:03,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,360 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:56:03,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:03,361 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:56:03,361 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:03,361 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:56:03,361 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:56:03,361 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:03,361 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:03,361 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:56:03,361 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:56:03,772 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037341dd379fca426d9cdb3f06d660c44d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601037341dd379fca426d9cdb3f06d660c44d 2023-02-08 23:56:04,177 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/77e6257456eb4c279abf66ceb2ba54a7 2023-02-08 23:56:04,181 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/77e6257456eb4c279abf66ceb2ba54a7 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/77e6257456eb4c279abf66ceb2ba54a7 2023-02-08 23:56:04,185 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/77e6257456eb4c279abf66ceb2ba54a7, entries=1, sequenceid=33, filesize=4.9 K 2023-02-08 23:56:04,185 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 824ms, sequenceid=33, compaction requested=true 2023-02-08 23:56:04,186 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:04,186 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:04,186 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:56:04,186 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:56:04,186 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,186 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:56:04,186 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:56:04,188 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,188 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:04,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:04,188 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,188 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:04,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:04,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:04,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:04,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:56:04,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53218bb9[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:04,190 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:56:04,190 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,192 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,192 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:56:04,192 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:56:04,192 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:04,192 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,192 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:04,193 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:56:04,193 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,193 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:04,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:04,193 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:56:04,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:04,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:04,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:56:04,196 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:56:04,196 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:04,196 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:04,196 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,196 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:04,197 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,197 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,198 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:04,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:04,198 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:04,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,199 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,204 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,204 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:04,204 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,204 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:04,204 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:04,204 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:04,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:04,204 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:04,204 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:04,204 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,204 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:04,205 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:04,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:04,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:04,204 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,205 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,205 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,305 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:04,305 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:04,305 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:04,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:04,309 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:04,309 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:04,310 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:04,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:04,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,311 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:04,311 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:04,311 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:56:04,311 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:56:04,312 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,312 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,313 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:04,313 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:04,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:04,313 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:04,314 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:04,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,314 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:04,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,314 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,315 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:56:04,315 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:04,315 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:56:04,315 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:04,315 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:56:04,315 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:56:04,315 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:04,315 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:04,315 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:56:04,315 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:56:04,727 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090795cce265958d4d479aeae3a7e860e036 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015090795cce265958d4d479aeae3a7e860e036 2023-02-08 23:56:05,132 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/46a849f95d384196b36a2a58536d0f2d 2023-02-08 23:56:05,136 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/46a849f95d384196b36a2a58536d0f2d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/46a849f95d384196b36a2a58536d0f2d 2023-02-08 23:56:05,139 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/46a849f95d384196b36a2a58536d0f2d, entries=1, sequenceid=37, filesize=4.9 K 2023-02-08 23:56:05,140 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 825ms, sequenceid=37, compaction requested=true 2023-02-08 23:56:05,140 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:05,140 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:05,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:56:05,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:56:05,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:56:05,140 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:56:05,141 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,142 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:05,142 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,142 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:05,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:56:05,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54582adc[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:05,143 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:56:05,143 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,144 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,144 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:56:05,144 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:56:05,144 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:05,144 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,144 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:05,147 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:56:05,147 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,147 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:05,147 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:56:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:05,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:05,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:05,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:05,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:56:05,150 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:56:05,150 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:05,150 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:05,150 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,150 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:05,151 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,151 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:05,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:05,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,151 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:05,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:05,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:05,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:05,152 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:05,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,153 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:05,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:05,155 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:05,155 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:05,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:05,155 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:56:05,155 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,155 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:05,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:05,256 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:05,256 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:05,256 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:05,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:05,260 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:05,260 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:05,261 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:05,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:05,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,263 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:05,263 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:05,263 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 23:56:05,263 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 23:56:05,263 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,263 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,264 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,264 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:05,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:05,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:05,264 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:05,264 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:05,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:05,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,265 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 23:56:05,265 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 23:56:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 23:56:05,266 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 23:56:05,266 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:05,266 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:05,266 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. started... 2023-02-08 23:56:05,266 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 56b1dbb83cb5199116335c5260eadbb4 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 23:56:05,678 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120af32452a859543ab84e919ddda61ac08 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120af32452a859543ab84e919ddda61ac08 2023-02-08 23:56:06,083 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/432572820bf84954a21f18867eb48580 2023-02-08 23:56:06,087 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/432572820bf84954a21f18867eb48580 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/432572820bf84954a21f18867eb48580 2023-02-08 23:56:06,090 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/432572820bf84954a21f18867eb48580, entries=1, sequenceid=41, filesize=4.9 K 2023-02-08 23:56:06,091 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 56b1dbb83cb5199116335c5260eadbb4 in 825ms, sequenceid=41, compaction requested=true 2023-02-08 23:56:06,091 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:06,091 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:06,091 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 23:56:06,091 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 23:56:06,091 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,091 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 23:56:06,091 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 23:56:06,094 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,094 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:06,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:06,094 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,094 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:06,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:06,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:06,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:06,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 23:56:06,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@400af32a[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:06,095 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 23:56:06,095 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,096 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,097 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 23:56:06,097 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 23:56:06,097 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:06,097 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,097 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:06,098 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 23:56:06,098 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,098 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:06,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:06,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:06,098 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 23:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:06,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 23:56:06,100 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 23:56:06,100 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:06,101 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:06,101 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,101 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:06,102 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,102 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,102 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:06,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:06,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:06,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:06,103 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:06,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:06,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:06,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,104 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:06,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:06,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,106 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:06,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 23:56:06,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:06,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:06,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:06,107 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 23:56:06,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:06,107 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:06,107 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:06,106 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:06,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:06,107 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:06,107 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,107 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,207 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:06,207 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:06,207 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:06,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:06,209 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-08 23:56:06,210 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:56:06,211 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:56:06,212 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:56:06,212 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901166212 2023-02-08 23:56:06,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 23:56:06,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-08 23:56:06,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:56:06,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x7e2f473c to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:56:06,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1eab0e13, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:56:06,219 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:56:07,043 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d30694bae95b411a96de0d3ea8615710 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103d30694bae95b411a96de0d3ea8615710 2023-02-08 23:56:07,044 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:56:07,057 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:56:07,059 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57006, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:56:07,064 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:07,064 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:07,064 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:56:07,065 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57284, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:56:07,075 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cbe8eb0e79614ee38ee7ad6a9a493016 first=Optional[r5] last=Optional[r6] 2023-02-08 23:56:07,076 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cbe8eb0e79614ee38ee7ad6a9a493016}] 2023-02-08 23:56:07,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cbe8eb0e79614ee38ee7ad6a9a493016 for inclusion in 56b1dbb83cb5199116335c5260eadbb4/family1 2023-02-08 23:56:07,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-08 23:56:07,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:56:07,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:07,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cbe8eb0e79614ee38ee7ad6a9a493016 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ol5ase6rcd1mubshau4nj0pvjo7h1rmp4nosorerf09ohnp05gpn7g0o6l5o6rkh/family1/cbe8eb0e79614ee38ee7ad6a9a493016 2023-02-08 23:56:07,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ol5ase6rcd1mubshau4nj0pvjo7h1rmp4nosorerf09ohnp05gpn7g0o6l5o6rkh/family1/cbe8eb0e79614ee38ee7ad6a9a493016 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/076d28b0d1334ebca139d4ecdf2edc21_SeqId_44_ 2023-02-08 23:56:07,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ol5ase6rcd1mubshau4nj0pvjo7h1rmp4nosorerf09ohnp05gpn7g0o6l5o6rkh/family1/cbe8eb0e79614ee38ee7ad6a9a493016 into 56b1dbb83cb5199116335c5260eadbb4/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/076d28b0d1334ebca139d4ecdf2edc21_SeqId_44_ - updating store file list. 2023-02-08 23:56:07,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/076d28b0d1334ebca139d4ecdf2edc21_SeqId_44_ into 56b1dbb83cb5199116335c5260eadbb4/family1 2023-02-08 23:56:07,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ol5ase6rcd1mubshau4nj0pvjo7h1rmp4nosorerf09ohnp05gpn7g0o6l5o6rkh/family1/cbe8eb0e79614ee38ee7ad6a9a493016 into 56b1dbb83cb5199116335c5260eadbb4/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/076d28b0d1334ebca139d4ecdf2edc21_SeqId_44_) 2023-02-08 23:56:07,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ol5ase6rcd1mubshau4nj0pvjo7h1rmp4nosorerf09ohnp05gpn7g0o6l5o6rkh/family1/cbe8eb0e79614ee38ee7ad6a9a493016 2023-02-08 23:56:07,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 23:56:07,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 23:56:07,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 56b1dbb83cb5199116335c5260eadbb4/family1 is initiating minor compaction 2023-02-08 23:56:07,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@33a57f3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:56:07,094 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 56b1dbb83cb5199116335c5260eadbb4/family1 in testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:07,094 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/5f8493689de541b5899e30053d800b5a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/f9371b84ab93440eae41a54d7f6ac91d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65f8adc54869482c858f3425761a5bc5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/40a1ae784aa042f994748fc316a98782, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/b9a9d786dc5f42548f11ce4a2421a1c1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/398ace704e6b4f1d887932275813fc28, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/77e6257456eb4c279abf66ceb2ba54a7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/46a849f95d384196b36a2a58536d0f2d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/432572820bf84954a21f18867eb48580, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/076d28b0d1334ebca139d4ecdf2edc21_SeqId_44_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp, totalSize=49.7 K 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 5f8493689de541b5899e30053d800b5a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting f9371b84ab93440eae41a54d7f6ac91d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 65f8adc54869482c858f3425761a5bc5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 40a1ae784aa042f994748fc316a98782, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting b9a9d786dc5f42548f11ce4a2421a1c1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 398ace704e6b4f1d887932275813fc28, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 77e6257456eb4c279abf66ceb2ba54a7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 46a849f95d384196b36a2a58536d0f2d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 432572820bf84954a21f18867eb48580, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 23:56:07,094 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 076d28b0d1334ebca139d4ecdf2edc21_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 23:56:07,097 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:56:07,097 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:07,099 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101c2dfb292283b453391d2cf99f6ffe3f8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101c2dfb292283b453391d2cf99f6ffe3f8 2023-02-08 23:56:07,100 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601037341dd379fca426d9cdb3f06d660c44d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601037341dd379fca426d9cdb3f06d660c44d 2023-02-08 23:56:07,100 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-08 23:56:07,100 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-08 23:56:07,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:56:07,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:56:07,120 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 56b1dbb83cb5199116335c5260eadbb4#family1#compaction#165 average throughput is 0.27 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:56:07,239 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:56:07,239 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e13a0e8 to 127.0.0.1:54774 2023-02-08 23:56:07,239 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:56:07,239 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:56:07,239 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2c3f2136 to 127.0.0.1:54774 2023-02-08 23:56:07,239 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:56:07,938 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/65fda8aa35544e1f96eac351fee7275a as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65fda8aa35544e1f96eac351fee7275a 2023-02-08 23:56:07,943 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 56b1dbb83cb5199116335c5260eadbb4/family1 of 56b1dbb83cb5199116335c5260eadbb4 into 65fda8aa35544e1f96eac351fee7275a(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:56:07,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:07,943 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675900567094; duration=0sec 2023-02-08 23:56:07,943 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:56:07,949 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512313350580db2044d27a8a97a2ea1fe118f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512313350580db2044d27a8a97a2ea1fe118f 2023-02-08 23:56:07,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:56:07,965 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:07,965 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:07,971 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/5f83caf1fcc743f0bc1f024757a448f9 first=Optional[r1] last=Optional[r4] 2023-02-08 23:56:07,972 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/5f83caf1fcc743f0bc1f024757a448f9}] 2023-02-08 23:56:07,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/5f83caf1fcc743f0bc1f024757a448f9 for inclusion in 56b1dbb83cb5199116335c5260eadbb4/family1 2023-02-08 23:56:07,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-08 23:56:07,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:56:07,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:07,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/5f83caf1fcc743f0bc1f024757a448f9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ff0hpndp2ispsbmmeoejriqnicl7tns8midf3sbnj9lp75tgjc5p5dvv8ksvqclh/family1/5f83caf1fcc743f0bc1f024757a448f9 2023-02-08 23:56:07,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ff0hpndp2ispsbmmeoejriqnicl7tns8midf3sbnj9lp75tgjc5p5dvv8ksvqclh/family1/5f83caf1fcc743f0bc1f024757a448f9 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/05b3d1fb6d504fdfa0e502801c153400_SeqId_47_ 2023-02-08 23:56:07,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ff0hpndp2ispsbmmeoejriqnicl7tns8midf3sbnj9lp75tgjc5p5dvv8ksvqclh/family1/5f83caf1fcc743f0bc1f024757a448f9 into 56b1dbb83cb5199116335c5260eadbb4/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/05b3d1fb6d504fdfa0e502801c153400_SeqId_47_ - updating store file list. 2023-02-08 23:56:07,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/05b3d1fb6d504fdfa0e502801c153400_SeqId_47_ into 56b1dbb83cb5199116335c5260eadbb4/family1 2023-02-08 23:56:07,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ff0hpndp2ispsbmmeoejriqnicl7tns8midf3sbnj9lp75tgjc5p5dvv8ksvqclh/family1/5f83caf1fcc743f0bc1f024757a448f9 into 56b1dbb83cb5199116335c5260eadbb4/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/05b3d1fb6d504fdfa0e502801c153400_SeqId_47_) 2023-02-08 23:56:07,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ff0hpndp2ispsbmmeoejriqnicl7tns8midf3sbnj9lp75tgjc5p5dvv8ksvqclh/family1/5f83caf1fcc743f0bc1f024757a448f9 2023-02-08 23:56:07,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 23:56:07,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 23:56:07,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(1912): 56b1dbb83cb5199116335c5260eadbb4/family1 is initiating minor compaction (all files) 2023-02-08 23:56:07,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1cd8ef19; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:56:07,983 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 56b1dbb83cb5199116335c5260eadbb4/family1 in testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:07,983 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/74c56c3666464462a505e6cfe365ee7d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65fda8aa35544e1f96eac351fee7275a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/05b3d1fb6d504fdfa0e502801c153400_SeqId_47_] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp, totalSize=178.1 K 2023-02-08 23:56:07,983 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 74c56c3666464462a505e6cfe365ee7d, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878838754 2023-02-08 23:56:07,983 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 65fda8aa35544e1f96eac351fee7275a, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621238754 2023-02-08 23:56:07,984 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] compactions.Compactor(207): Compacting 05b3d1fb6d504fdfa0e502801c153400_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965238754 2023-02-08 23:56:07,984 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:56:07,985 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:07,986 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201270600c2096748b59ce163a7648094be to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201270600c2096748b59ce163a7648094be 2023-02-08 23:56:07,987 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205147dd31ea6aa41bc8ec0ff157399283d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205147dd31ea6aa41bc8ec0ff157399283d 2023-02-08 23:56:07,988 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228d39e097da81645db8479344ccfe2cef8 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228d39e097da81645db8479344ccfe2cef8 2023-02-08 23:56:07,989 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231d6f1d95ce88043d4819c73d11e907121 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231d6f1d95ce88043d4819c73d11e907121 2023-02-08 23:56:07,989 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-08 23:56:07,989 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-08 23:56:07,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:56:07,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:56:07,996 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] throttle.PressureAwareThroughputController(145): 56b1dbb83cb5199116335c5260eadbb4#family1#compaction#166 average throughput is 22.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:56:08,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/5f8493689de541b5899e30053d800b5a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/f9371b84ab93440eae41a54d7f6ac91d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65f8adc54869482c858f3425761a5bc5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/40a1ae784aa042f994748fc316a98782, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/b9a9d786dc5f42548f11ce4a2421a1c1, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/398ace704e6b4f1d887932275813fc28, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/77e6257456eb4c279abf66ceb2ba54a7, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/46a849f95d384196b36a2a58536d0f2d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/432572820bf84954a21f18867eb48580, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/076d28b0d1334ebca139d4ecdf2edc21_SeqId_44_] to archive 2023-02-08 23:56:08,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:08,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/5f8493689de541b5899e30053d800b5a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/5f8493689de541b5899e30053d800b5a 2023-02-08 23:56:08,031 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/f9371b84ab93440eae41a54d7f6ac91d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/f9371b84ab93440eae41a54d7f6ac91d 2023-02-08 23:56:08,033 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65f8adc54869482c858f3425761a5bc5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65f8adc54869482c858f3425761a5bc5 2023-02-08 23:56:08,034 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/40a1ae784aa042f994748fc316a98782 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/40a1ae784aa042f994748fc316a98782 2023-02-08 23:56:08,035 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/b9a9d786dc5f42548f11ce4a2421a1c1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/b9a9d786dc5f42548f11ce4a2421a1c1 2023-02-08 23:56:08,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/398ace704e6b4f1d887932275813fc28 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/398ace704e6b4f1d887932275813fc28 2023-02-08 23:56:08,037 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/77e6257456eb4c279abf66ceb2ba54a7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/77e6257456eb4c279abf66ceb2ba54a7 2023-02-08 23:56:08,037 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/46a849f95d384196b36a2a58536d0f2d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/46a849f95d384196b36a2a58536d0f2d 2023-02-08 23:56:08,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/432572820bf84954a21f18867eb48580 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/432572820bf84954a21f18867eb48580 2023-02-08 23:56:08,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/076d28b0d1334ebca139d4ecdf2edc21_SeqId_44_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/076d28b0d1334ebca139d4ecdf2edc21_SeqId_44_ 2023-02-08 23:56:08,420 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130adcc20d4557a456e842d28ef06b21c49 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130adcc20d4557a456e842d28ef06b21c49 2023-02-08 23:56:08,421 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:56:08,436 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:08,437 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:08,443 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/138b139074b14fe694c73e014766b09d first=Optional[r00] last=Optional[r8] 2023-02-08 23:56:08,444 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/138b139074b14fe694c73e014766b09d}] 2023-02-08 23:56:08,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/138b139074b14fe694c73e014766b09d for inclusion in 56b1dbb83cb5199116335c5260eadbb4/family1 2023-02-08 23:56:08,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-08 23:56:08,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 23:56:08,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:08,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/138b139074b14fe694c73e014766b09d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9eb3sn46ebg151ne2pkaffiuhkjpkaemrqi4r3hna1lhv0tl4tnc2durrmvnv335/family1/138b139074b14fe694c73e014766b09d 2023-02-08 23:56:08,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9eb3sn46ebg151ne2pkaffiuhkjpkaemrqi4r3hna1lhv0tl4tnc2durrmvnv335/family1/138b139074b14fe694c73e014766b09d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/cd5e3195bf63489884ac8a7b50bf733c_SeqId_49_ 2023-02-08 23:56:08,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9eb3sn46ebg151ne2pkaffiuhkjpkaemrqi4r3hna1lhv0tl4tnc2durrmvnv335/family1/138b139074b14fe694c73e014766b09d into 56b1dbb83cb5199116335c5260eadbb4/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/cd5e3195bf63489884ac8a7b50bf733c_SeqId_49_ - updating store file list. 2023-02-08 23:56:08,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/cd5e3195bf63489884ac8a7b50bf733c_SeqId_49_ into 56b1dbb83cb5199116335c5260eadbb4/family1 2023-02-08 23:56:08,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9eb3sn46ebg151ne2pkaffiuhkjpkaemrqi4r3hna1lhv0tl4tnc2durrmvnv335/family1/138b139074b14fe694c73e014766b09d into 56b1dbb83cb5199116335c5260eadbb4/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/cd5e3195bf63489884ac8a7b50bf733c_SeqId_49_) 2023-02-08 23:56:08,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMinorCompactionWithMonthlyPolicy__9eb3sn46ebg151ne2pkaffiuhkjpkaemrqi4r3hna1lhv0tl4tnc2durrmvnv335/family1/138b139074b14fe694c73e014766b09d 2023-02-08 23:56:08,458 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:56:08,458 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:08,459 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120af32452a859543ab84e919ddda61ac08 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120af32452a859543ab84e919ddda61ac08 2023-02-08 23:56:08,460 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128fc4e102addcb406bb8f94bc38696c696 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128fc4e102addcb406bb8f94bc38696c696 2023-02-08 23:56:08,461 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113002ec42eb488747e38c16593e8450a192 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113002ec42eb488747e38c16593e8450a192 2023-02-08 23:56:08,461 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-08 23:56:08,461 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 23:56:08,462 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:56:08,462 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:56:08,462 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 23:56:08,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 23:56:08,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:56:08,464 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 23:56:08,465 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.2540 sec 2023-02-08 23:56:08,665 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=257 (was 252) - Thread LEAK? -, OpenFileDescriptor=652 (was 645) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=100 (was 100), ProcessCount=168 (was 167) - ProcessCount LEAK? -, AvailableMemoryMB=3284 (was 3384) 2023-02-08 23:56:08,676 INFO [Listener at localhost/41521] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=257, OpenFileDescriptor=652, MaxFileDescriptor=60000, SystemLoadAverage=100, ProcessCount=168, AvailableMemoryMB=3283 2023-02-08 23:56:08,696 DEBUG [Listener at localhost/41521] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-08 23:56:08,719 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:08,719 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:08,751 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCA*f{\xCF\x9E\xB3/W\xB2\xAF\x14\xEF\x18j\xBF"\x10\x9Fy\x8B~\xACP\xA0}\x84pVGZ\xCA\x989*\x10\x9C\x0A\xAC\xEBA\x0B\xA8\x0E\x1C\x04Zu`\xE2#\xD12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 23:56:08,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-08 23:56:08,754 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 23:56:08,754 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-08 23:56:08,755 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:08,755 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 23:56:08,755 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:08,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 23:56:08,757 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:08,759 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376 empty. 2023-02-08 23:56:08,759 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:08,759 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:08,759 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:08,759 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8 empty. 2023-02-08 23:56:08,759 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a empty. 2023-02-08 23:56:08,760 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:08,760 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:08,760 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-08 23:56:08,856 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:08,856 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:08,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 23:56:09,057 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:09,057 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:09,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 23:56:09,169 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-08 23:56:09,171 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => f0c73908a5b8d0551609b98f0efe640a, NAME => 'testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCA*f{\xCF\x9E\xB3/W\xB2\xAF\x14\xEF\x18j\xBF"\x10\x9Fy\x8B~\xACP\xA0}\x84pVGZ\xCA\x989*\x10\x9C\x0A\xAC\xEBA\x0B\xA8\x0E\x1C\x04Zu`\xE2#\xD12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:56:09,171 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => ab31217c6a226641e72afe8624c897d8, NAME => 'testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCA*f{\xCF\x9E\xB3/W\xB2\xAF\x14\xEF\x18j\xBF"\x10\x9Fy\x8B~\xACP\xA0}\x84pVGZ\xCA\x989*\x10\x9C\x0A\xAC\xEBA\x0B\xA8\x0E\x1C\x04Zu`\xE2#\xD12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:56:09,171 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => a3bc650bd8f95f4b16a947f7e690d376, NAME => 'testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCA*f{\xCF\x9E\xB3/W\xB2\xAF\x14\xEF\x18j\xBF"\x10\x9Fy\x8B~\xACP\xA0}\x84pVGZ\xCA\x989*\x10\x9C\x0A\xAC\xEBA\x0B\xA8\x0E\x1C\x04Zu`\xE2#\xD12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/.tmp 2023-02-08 23:56:09,182 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:56:09,182 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing f0c73908a5b8d0551609b98f0efe640a, disabling compactions & flushes 2023-02-08 23:56:09,182 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:09,182 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:09,182 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. after waiting 0 ms 2023-02-08 23:56:09,182 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:09,182 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:09,182 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for f0c73908a5b8d0551609b98f0efe640a: Waiting for close lock at 1675900569182Disabling compacts and flushes for region at 1675900569182Disabling writes for close at 1675900569182Writing region close event to WAL at 1675900569182Closed at 1675900569182 2023-02-08 23:56:09,183 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:56:09,183 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing ab31217c6a226641e72afe8624c897d8, disabling compactions & flushes 2023-02-08 23:56:09,184 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:09,184 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:09,184 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. after waiting 0 ms 2023-02-08 23:56:09,184 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:09,184 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:09,184 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for ab31217c6a226641e72afe8624c897d8: Waiting for close lock at 1675900569183Disabling compacts and flushes for region at 1675900569183Disabling writes for close at 1675900569184 (+1 ms)Writing region close event to WAL at 1675900569184Closed at 1675900569184 2023-02-08 23:56:09,211 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/.tmp/family1/1b1b1fbb18b041f484657816cba19c5f as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/1b1b1fbb18b041f484657816cba19c5f 2023-02-08 23:56:09,215 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 56b1dbb83cb5199116335c5260eadbb4/family1 of 56b1dbb83cb5199116335c5260eadbb4 into 1b1b1fbb18b041f484657816cba19c5f(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:56:09,215 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 56b1dbb83cb5199116335c5260eadbb4: 2023-02-08 23:56:09,215 INFO [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1675900567982; duration=1sec 2023-02-08 23:56:09,215 DEBUG [RS:0;jenkins-hbase4:42703-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 23:56:09,226 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/74c56c3666464462a505e6cfe365ee7d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65fda8aa35544e1f96eac351fee7275a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/05b3d1fb6d504fdfa0e502801c153400_SeqId_47_] to archive 2023-02-08 23:56:09,227 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:09,228 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/74c56c3666464462a505e6cfe365ee7d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/74c56c3666464462a505e6cfe365ee7d 2023-02-08 23:56:09,229 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65fda8aa35544e1f96eac351fee7275a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/65fda8aa35544e1f96eac351fee7275a 2023-02-08 23:56:09,230 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/05b3d1fb6d504fdfa0e502801c153400_SeqId_47_ to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/family1/05b3d1fb6d504fdfa0e502801c153400_SeqId_47_ 2023-02-08 23:56:09,358 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:09,358 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:09,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 23:56:09,583 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:56:09,583 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing a3bc650bd8f95f4b16a947f7e690d376, disabling compactions & flushes 2023-02-08 23:56:09,583 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:09,583 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:09,583 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. after waiting 0 ms 2023-02-08 23:56:09,583 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:09,583 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:09,583 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for a3bc650bd8f95f4b16a947f7e690d376: Waiting for close lock at 1675900569583Disabling compacts and flushes for region at 1675900569583Disabling writes for close at 1675900569583Writing region close event to WAL at 1675900569583Closed at 1675900569583 2023-02-08 23:56:09,585 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 23:56:09,586 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900569586"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900569586"}]},"ts":"1675900569586"} 2023-02-08 23:56:09,586 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900569586"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900569586"}]},"ts":"1675900569586"} 2023-02-08 23:56:09,586 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900569586"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675900569586"}]},"ts":"1675900569586"} 2023-02-08 23:56:09,588 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 23:56:09,588 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 23:56:09,588 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900569588"}]},"ts":"1675900569588"} 2023-02-08 23:56:09,589 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-08 23:56:09,592 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a3bc650bd8f95f4b16a947f7e690d376, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ab31217c6a226641e72afe8624c897d8, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f0c73908a5b8d0551609b98f0efe640a, ASSIGN}] 2023-02-08 23:56:09,594 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a3bc650bd8f95f4b16a947f7e690d376, ASSIGN 2023-02-08 23:56:09,594 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f0c73908a5b8d0551609b98f0efe640a, ASSIGN 2023-02-08 23:56:09,594 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ab31217c6a226641e72afe8624c897d8, ASSIGN 2023-02-08 23:56:09,594 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a3bc650bd8f95f4b16a947f7e690d376, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:56:09,594 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f0c73908a5b8d0551609b98f0efe640a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:56:09,594 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ab31217c6a226641e72afe8624c897d8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42703,1675900414768; forceNewPlan=false, retain=false 2023-02-08 23:56:09,746 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=f0c73908a5b8d0551609b98f0efe640a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:09,746 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=a3bc650bd8f95f4b16a947f7e690d376, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:09,746 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900569746"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900569746"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900569746"}]},"ts":"1675900569746"} 2023-02-08 23:56:09,746 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ab31217c6a226641e72afe8624c897d8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:09,746 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900569746"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900569746"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900569746"}]},"ts":"1675900569746"} 2023-02-08 23:56:09,746 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900569746"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675900569746"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675900569746"}]},"ts":"1675900569746"} 2023-02-08 23:56:09,748 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE; OpenRegionProcedure f0c73908a5b8d0551609b98f0efe640a, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:56:09,748 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure ab31217c6a226641e72afe8624c897d8, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:56:09,748 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=117, state=RUNNABLE; OpenRegionProcedure a3bc650bd8f95f4b16a947f7e690d376, server=jenkins-hbase4.apache.org,42703,1675900414768}] 2023-02-08 23:56:09,859 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:09,859 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:09,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 23:56:09,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:09,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ab31217c6a226641e72afe8624c897d8, NAME => 'testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 23:56:09,902 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:09,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:56:09,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:09,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:09,924 INFO [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:09,925 DEBUG [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1 2023-02-08 23:56:09,925 DEBUG [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1 2023-02-08 23:56:09,925 INFO [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab31217c6a226641e72afe8624c897d8 columnFamilyName family1 2023-02-08 23:56:09,926 INFO [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] regionserver.HStore(310): Store=ab31217c6a226641e72afe8624c897d8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:56:09,926 INFO [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:09,927 DEBUG [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2 2023-02-08 23:56:09,927 DEBUG [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2 2023-02-08 23:56:09,928 INFO [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab31217c6a226641e72afe8624c897d8 columnFamilyName family2 2023-02-08 23:56:09,928 INFO [StoreOpener-ab31217c6a226641e72afe8624c897d8-1] regionserver.HStore(310): Store=ab31217c6a226641e72afe8624c897d8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:56:09,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:09,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:09,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:56:09,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:09,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:56:09,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ab31217c6a226641e72afe8624c897d8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67668989, jitterRate=0.008346512913703918}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:56:09,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ab31217c6a226641e72afe8624c897d8: Running coprocessor pre-open hook at 1675900569903Writing region info on filesystem at 1675900569903Initializing all the Stores at 1675900569903Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCA*f{\xCF\x9E\xB3/W\xB2\xAF\x14\xEF\x18j\xBF"\x10\x9Fy\x8B~\xACP\xA0}\x84pVGZ\xCA\x989*\x10\x9C\x0A\xAC\xEBA\x0B\xA8\x0E\x1C\x04Zu`\xE2#\xD12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900569903Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900569903Cleaning up temporary data from old regions at 1675900569930 (+27 ms)Cleaning up detritus from prior splits at 1675900569930Running coprocessor post-open hooks at 1675900569933 (+3 ms)Region opened successfully at 1675900569933 2023-02-08 23:56:09,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8., pid=121, masterSystemTime=1675900569899 2023-02-08 23:56:09,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:09,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:09,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:09,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a3bc650bd8f95f4b16a947f7e690d376, NAME => 'testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 23:56:09,935 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ab31217c6a226641e72afe8624c897d8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:09,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:09,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:56:09,936 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675900569935"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900569935"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900569935"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900569935"}]},"ts":"1675900569935"} 2023-02-08 23:56:09,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:09,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:09,938 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-02-08 23:56:09,938 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure ab31217c6a226641e72afe8624c897d8, server=jenkins-hbase4.apache.org,42703,1675900414768 in 189 msec 2023-02-08 23:56:09,939 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ab31217c6a226641e72afe8624c897d8, ASSIGN in 346 msec 2023-02-08 23:56:09,966 INFO [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:09,967 DEBUG [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1 2023-02-08 23:56:09,967 DEBUG [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1 2023-02-08 23:56:09,967 INFO [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3bc650bd8f95f4b16a947f7e690d376 columnFamilyName family1 2023-02-08 23:56:09,968 INFO [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] regionserver.HStore(310): Store=a3bc650bd8f95f4b16a947f7e690d376/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:56:09,968 INFO [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:09,969 DEBUG [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2 2023-02-08 23:56:09,970 DEBUG [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2 2023-02-08 23:56:09,970 INFO [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3bc650bd8f95f4b16a947f7e690d376 columnFamilyName family2 2023-02-08 23:56:09,970 INFO [StoreOpener-a3bc650bd8f95f4b16a947f7e690d376-1] regionserver.HStore(310): Store=a3bc650bd8f95f4b16a947f7e690d376/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:56:09,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:09,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:09,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:56:09,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:09,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:56:09,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a3bc650bd8f95f4b16a947f7e690d376; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65968142, jitterRate=-0.016998082399368286}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:56:09,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a3bc650bd8f95f4b16a947f7e690d376: Running coprocessor pre-open hook at 1675900569936Writing region info on filesystem at 1675900569936Initializing all the Stores at 1675900569936Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCA*f{\xCF\x9E\xB3/W\xB2\xAF\x14\xEF\x18j\xBF"\x10\x9Fy\x8B~\xACP\xA0}\x84pVGZ\xCA\x989*\x10\x9C\x0A\xAC\xEBA\x0B\xA8\x0E\x1C\x04Zu`\xE2#\xD12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900569937 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900569937Cleaning up temporary data from old regions at 1675900569972 (+35 ms)Cleaning up detritus from prior splits at 1675900569972Running coprocessor post-open hooks at 1675900569976 (+4 ms)Region opened successfully at 1675900569976 2023-02-08 23:56:09,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376., pid=122, masterSystemTime=1675900569899 2023-02-08 23:56:09,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:09,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:09,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:09,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f0c73908a5b8d0551609b98f0efe640a, NAME => 'testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 23:56:09,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:09,978 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=a3bc650bd8f95f4b16a947f7e690d376, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:09,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 23:56:09,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:09,978 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900569978"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900569978"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900569978"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900569978"}]},"ts":"1675900569978"} 2023-02-08 23:56:09,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:09,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=117 2023-02-08 23:56:09,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=117, state=SUCCESS; OpenRegionProcedure a3bc650bd8f95f4b16a947f7e690d376, server=jenkins-hbase4.apache.org,42703,1675900414768 in 231 msec 2023-02-08 23:56:09,982 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a3bc650bd8f95f4b16a947f7e690d376, ASSIGN in 389 msec 2023-02-08 23:56:10,017 INFO [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:10,018 DEBUG [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1 2023-02-08 23:56:10,018 DEBUG [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1 2023-02-08 23:56:10,019 INFO [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0c73908a5b8d0551609b98f0efe640a columnFamilyName family1 2023-02-08 23:56:10,019 INFO [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] regionserver.HStore(310): Store=f0c73908a5b8d0551609b98f0efe640a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:56:10,020 INFO [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:10,021 DEBUG [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2 2023-02-08 23:56:10,021 DEBUG [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2 2023-02-08 23:56:10,021 INFO [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0c73908a5b8d0551609b98f0efe640a columnFamilyName family2 2023-02-08 23:56:10,022 INFO [StoreOpener-f0c73908a5b8d0551609b98f0efe640a-1] regionserver.HStore(310): Store=f0c73908a5b8d0551609b98f0efe640a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 23:56:10,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:10,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:10,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 23:56:10,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:10,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 23:56:10,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f0c73908a5b8d0551609b98f0efe640a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70812700, jitterRate=0.05519145727157593}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 23:56:10,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f0c73908a5b8d0551609b98f0efe640a: Running coprocessor pre-open hook at 1675900569978Writing region info on filesystem at 1675900569978Initializing all the Stores at 1675900569979 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCA*f{\xCF\x9E\xB3/W\xB2\xAF\x14\xEF\x18j\xBF"\x10\x9Fy\x8B~\xACP\xA0}\x84pVGZ\xCA\x989*\x10\x9C\x0A\xAC\xEBA\x0B\xA8\x0E\x1C\x04Zu`\xE2#\xD12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900569979Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675900569988 (+9 ms)Cleaning up temporary data from old regions at 1675900570023 (+35 ms)Cleaning up detritus from prior splits at 1675900570023Running coprocessor post-open hooks at 1675900570027 (+4 ms)Region opened successfully at 1675900570027 2023-02-08 23:56:10,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a., pid=120, masterSystemTime=1675900569899 2023-02-08 23:56:10,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:10,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:10,029 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=f0c73908a5b8d0551609b98f0efe640a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:10,029 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675900570029"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675900570029"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675900570029"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675900570029"}]},"ts":"1675900570029"} 2023-02-08 23:56:10,032 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-08 23:56:10,032 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; OpenRegionProcedure f0c73908a5b8d0551609b98f0efe640a, server=jenkins-hbase4.apache.org,42703,1675900414768 in 283 msec 2023-02-08 23:56:10,033 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-02-08 23:56:10,033 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f0c73908a5b8d0551609b98f0efe640a, ASSIGN in 440 msec 2023-02-08 23:56:10,033 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 23:56:10,034 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675900570034"}]},"ts":"1675900570034"} 2023-02-08 23:56:10,035 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-08 23:56:10,037 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 23:56:10,038 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2850 sec 2023-02-08 23:56:10,860 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:10,861 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:10,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 23:56:10,862 INFO [Listener at localhost/41521] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-08 23:56:10,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:10,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:10,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:10,869 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:10,870 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:10,870 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:10,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 23:56:10,872 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 23:56:10,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:10,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 23:56:10,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 23:56:10,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:10,872 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:10,881 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:10,881 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:10,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:10,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:10,881 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:10,881 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:10,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:10,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:10,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:10,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:10,882 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 23:56:10,882 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 23:56:10,882 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 23:56:10,882 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:10,882 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 23:56:10,883 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:10,883 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:10,883 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:56:10,883 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:10,883 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. started... 2023-02-08 23:56:10,883 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:10,883 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. started... 2023-02-08 23:56:10,884 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing a3bc650bd8f95f4b16a947f7e690d376 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 23:56:10,883 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. started... 2023-02-08 23:56:10,885 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0c73908a5b8d0551609b98f0efe640a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 23:56:10,885 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ab31217c6a226641e72afe8624c897d8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 23:56:10,940 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302085ee50b3aac824da19d41887360469353 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302085ee50b3aac824da19d41887360469353 2023-02-08 23:56:10,940 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302086aa839aaf90b4daea7cf2ebc2e605058 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302086aa839aaf90b4daea7cf2ebc2e605058 2023-02-08 23:56:11,331 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208904593358edd4d0ca12671a4378861ac to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208904593358edd4d0ca12671a4378861ac 2023-02-08 23:56:11,355 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/1ec1500eb2d5405493775cf50ca4301c 2023-02-08 23:56:11,357 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/0165b15c056a4b14bc787da762d2d9f5 2023-02-08 23:56:11,388 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302081ec1c262bc9a4b4c927b1e087f74b4c7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302081ec1c262bc9a4b4c927b1e087f74b4c7 2023-02-08 23:56:11,392 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302086306b67a33f64e1ea4bc6a76ca85d493 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302086306b67a33f64e1ea4bc6a76ca85d493 2023-02-08 23:56:11,394 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/a3b3ca04283648629418ab49ec9e6dff 2023-02-08 23:56:11,397 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/71fb3bd8f6f147f2a9b1e509b0f780e0 2023-02-08 23:56:11,398 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/1ec1500eb2d5405493775cf50ca4301c as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/1ec1500eb2d5405493775cf50ca4301c 2023-02-08 23:56:11,401 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/0165b15c056a4b14bc787da762d2d9f5 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/0165b15c056a4b14bc787da762d2d9f5 2023-02-08 23:56:11,423 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/1ec1500eb2d5405493775cf50ca4301c, entries=4, sequenceid=5, filesize=5.5 K 2023-02-08 23:56:11,424 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/a3b3ca04283648629418ab49ec9e6dff as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/a3b3ca04283648629418ab49ec9e6dff 2023-02-08 23:56:11,424 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/0165b15c056a4b14bc787da762d2d9f5, entries=4, sequenceid=5, filesize=5.5 K 2023-02-08 23:56:11,425 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/71fb3bd8f6f147f2a9b1e509b0f780e0 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/71fb3bd8f6f147f2a9b1e509b0f780e0 2023-02-08 23:56:11,428 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/a3b3ca04283648629418ab49ec9e6dff, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 23:56:11,428 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/71fb3bd8f6f147f2a9b1e509b0f780e0, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 23:56:11,429 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f0c73908a5b8d0551609b98f0efe640a in 545ms, sequenceid=5, compaction requested=false 2023-02-08 23:56:11,429 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a3bc650bd8f95f4b16a947f7e690d376 in 545ms, sequenceid=5, compaction requested=false 2023-02-08 23:56:11,429 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-08 23:56:11,429 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-08 23:56:11,430 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0c73908a5b8d0551609b98f0efe640a: 2023-02-08 23:56:11,431 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for a3bc650bd8f95f4b16a947f7e690d376: 2023-02-08 23:56:11,431 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:11,431 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:11,431 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:56:11,431 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:56:11,747 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/6be73641c36f4217bf35767e4961c8e3 2023-02-08 23:56:12,182 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020860a749cebe1240b486480fc6f3bea7e1 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023020860a749cebe1240b486480fc6f3bea7e1 2023-02-08 23:56:12,586 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/491515f6e7a44b7fa68e957d2bc0a295 2023-02-08 23:56:12,590 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/6be73641c36f4217bf35767e4961c8e3 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/6be73641c36f4217bf35767e4961c8e3 2023-02-08 23:56:12,614 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/6be73641c36f4217bf35767e4961c8e3, entries=4, sequenceid=5, filesize=5.5 K 2023-02-08 23:56:12,615 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/491515f6e7a44b7fa68e957d2bc0a295 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/491515f6e7a44b7fa68e957d2bc0a295 2023-02-08 23:56:12,618 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/491515f6e7a44b7fa68e957d2bc0a295, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 23:56:12,619 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for ab31217c6a226641e72afe8624c897d8 in 1734ms, sequenceid=5, compaction requested=false 2023-02-08 23:56:12,619 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ab31217c6a226641e72afe8624c897d8: 2023-02-08 23:56:12,619 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:56:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:56:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 23:56:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 23:56:12,620 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,620 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:12,621 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:12,621 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:12,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 23:56:12,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72077095[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:12,622 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 23:56:12,622 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:12,623 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:12,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:12,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:12,623 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 23:56:12,623 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 23:56:12,623 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:12,623 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,623 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:12,624 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 23:56:12,624 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:12,624 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,624 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 23:56:12,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:12,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:12,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:12,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:12,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 23:56:12,626 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 23:56:12,626 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:12,626 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:12,626 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 23:56:12,626 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:12,634 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:12,634 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:12,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:12,634 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:12,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:12,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:12,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:12,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:12,634 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:12,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:12,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:12,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:12,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:12,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:12,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:12,635 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:12,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:12,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,637 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,637 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:12,637 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:12,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:12,637 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:12,637 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 23:56:12,637 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:12,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-08 23:56:12,637 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:12,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:12,638 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:12,638 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,638 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:12,638 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:12,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:12,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:12,638 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:12,638 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:12,638 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:12,738 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:12,738 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:12,738 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:12,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:12,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:12,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:12,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:12,742 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:12,742 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:12,743 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:12,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:12,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 23:56:12,745 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 23:56:12,745 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:12,745 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 23:56:12,745 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 23:56:12,745 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:12,745 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:12,746 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:12,746 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:12,747 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:12,747 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:12,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:12,748 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 23:56:12,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-08 23:56:12,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 23:56:12,748 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 23:56:12,748 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:12,748 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 23:56:12,748 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:12,748 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. started... 2023-02-08 23:56:12,748 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:12,748 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. started... 2023-02-08 23:56:12,748 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0c73908a5b8d0551609b98f0efe640a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 23:56:12,749 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing a3bc650bd8f95f4b16a947f7e690d376 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 23:56:12,750 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:56:12,751 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:12,751 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:12,751 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. started... 2023-02-08 23:56:12,751 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ab31217c6a226641e72afe8624c897d8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 23:56:13,198 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208eb77a26a45894668ba30287dfd6b1c46 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208eb77a26a45894668ba30287dfd6b1c46 2023-02-08 23:56:13,198 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302081b706508c1004ed6a2d0a7e660aabc9f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302081b706508c1004ed6a2d0a7e660aabc9f 2023-02-08 23:56:13,209 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020821e5d8cb14e54df2b1b630b4f32ef29b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020821e5d8cb14e54df2b1b630b4f32ef29b 2023-02-08 23:56:13,225 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/3bfa9a7a94294c489506ec21de5d54f4 2023-02-08 23:56:13,614 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/65cdfb10f8dc4cc695523e80c46c049d 2023-02-08 23:56:13,614 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/e8f71c67b2cd45f5bda4b6a3c26643e2 2023-02-08 23:56:13,654 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208b2ce83e7e1d144b99d6acfda31809ef2 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230208b2ce83e7e1d144b99d6acfda31809ef2 2023-02-08 23:56:13,666 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208d829f32ef3334983b32e2573a5a8b0b3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230208d829f32ef3334983b32e2573a5a8b0b3 2023-02-08 23:56:14,050 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208c20703e569654f0ea55870c05f6a670e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230208c20703e569654f0ea55870c05f6a670e 2023-02-08 23:56:14,059 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/ba34eebffdd946948e37bd26d1e95717 2023-02-08 23:56:14,063 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/e8f71c67b2cd45f5bda4b6a3c26643e2 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/e8f71c67b2cd45f5bda4b6a3c26643e2 2023-02-08 23:56:14,070 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/6f6ce1f80b1c4570a18c0b905671ef42 2023-02-08 23:56:14,074 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/3bfa9a7a94294c489506ec21de5d54f4 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/3bfa9a7a94294c489506ec21de5d54f4 2023-02-08 23:56:14,097 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/3bfa9a7a94294c489506ec21de5d54f4, entries=4, sequenceid=9, filesize=5.5 K 2023-02-08 23:56:14,097 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/6f6ce1f80b1c4570a18c0b905671ef42 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/6f6ce1f80b1c4570a18c0b905671ef42 2023-02-08 23:56:14,100 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/e8f71c67b2cd45f5bda4b6a3c26643e2, entries=4, sequenceid=9, filesize=5.5 K 2023-02-08 23:56:14,100 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/ba34eebffdd946948e37bd26d1e95717 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/ba34eebffdd946948e37bd26d1e95717 2023-02-08 23:56:14,101 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/6f6ce1f80b1c4570a18c0b905671ef42, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 23:56:14,102 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a3bc650bd8f95f4b16a947f7e690d376 in 1354ms, sequenceid=9, compaction requested=false 2023-02-08 23:56:14,102 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for a3bc650bd8f95f4b16a947f7e690d376: 2023-02-08 23:56:14,102 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:14,102 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:56:14,104 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/ba34eebffdd946948e37bd26d1e95717, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 23:56:14,104 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f0c73908a5b8d0551609b98f0efe640a in 1356ms, sequenceid=9, compaction requested=false 2023-02-08 23:56:14,104 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0c73908a5b8d0551609b98f0efe640a: 2023-02-08 23:56:14,104 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:14,104 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:56:14,455 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/e975b54b93eb4992876c6508ad8b2497 2023-02-08 23:56:14,459 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/65cdfb10f8dc4cc695523e80c46c049d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/65cdfb10f8dc4cc695523e80c46c049d 2023-02-08 23:56:14,483 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/65cdfb10f8dc4cc695523e80c46c049d, entries=4, sequenceid=9, filesize=5.5 K 2023-02-08 23:56:14,484 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/e975b54b93eb4992876c6508ad8b2497 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/e975b54b93eb4992876c6508ad8b2497 2023-02-08 23:56:14,488 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/e975b54b93eb4992876c6508ad8b2497, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 23:56:14,489 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for ab31217c6a226641e72afe8624c897d8 in 1738ms, sequenceid=9, compaction requested=false 2023-02-08 23:56:14,489 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ab31217c6a226641e72afe8624c897d8: 2023-02-08 23:56:14,489 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:14,489 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:56:14,489 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:56:14,489 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,489 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 23:56:14,489 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 23:56:14,491 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:14,491 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:14,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:14,491 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:14,491 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:14,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:14,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:14,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:14,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:14,493 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 23:56:14,493 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2da4412f[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:14,493 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 23:56:14,494 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:14,495 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:14,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:14,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:14,496 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 23:56:14,496 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 23:56:14,496 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:14,496 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,496 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:14,497 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 23:56:14,497 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:14,497 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,497 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 23:56:14,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:14,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:14,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:14,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:14,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:14,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:14,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:14,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:14,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 23:56:14,499 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 23:56:14,499 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:14,499 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:14,499 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 23:56:14,499 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:14,501 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:14,501 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:14,501 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:14,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:14,501 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:14,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:14,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:14,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:14,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:14,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:14,502 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:14,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:14,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:14,504 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,504 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:14,504 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:14,504 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,504 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:14,504 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:14,504 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:14,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:14,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:14,504 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 23:56:14,505 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:14,504 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:14,504 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:14,505 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:14,505 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:14,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:14,505 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:14,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:14,505 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-08 23:56:14,605 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:14,605 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:14,605 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:14,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:14,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:14,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:14,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:14,609 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:14,610 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:14,610 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:14,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:14,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 23:56:14,612 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 23:56:14,612 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:14,612 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 23:56:14,612 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 23:56:14,612 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:14,613 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:14,615 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,615 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:14,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:14,615 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:14,615 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:14,615 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:14,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:14,616 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 23:56:14,616 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-08 23:56:14,616 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 23:56:14,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 23:56:14,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:14,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 23:56:14,616 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:14,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:56:14,616 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:14,616 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:14,616 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. started... 2023-02-08 23:56:14,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:14,616 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. started... 2023-02-08 23:56:14,616 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. started... 2023-02-08 23:56:14,617 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ab31217c6a226641e72afe8624c897d8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 23:56:14,617 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0c73908a5b8d0551609b98f0efe640a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 23:56:14,617 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing a3bc650bd8f95f4b16a947f7e690d376 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 23:56:14,667 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208add635f9e8c24723997118e4a0974d71 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208add635f9e8c24723997118e4a0974d71 2023-02-08 23:56:14,673 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020894d6fb904b8a492ca70cf5dc00ef4989 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020894d6fb904b8a492ca70cf5dc00ef4989 2023-02-08 23:56:14,680 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302087bf74f671c6a4969a0a9426c8d478e65 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302087bf74f671c6a4969a0a9426c8d478e65 2023-02-08 23:56:14,694 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/4e7c811bbd654ecdb311d5f10fa3c262 2023-02-08 23:56:15,091 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/ddf1424cc68f42d8b6c8de25695ee44a 2023-02-08 23:56:15,108 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/ed3d16f9dceb41f2bc0c5c725c03978b 2023-02-08 23:56:15,133 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302087ea13cc3ad974405b316c6ade3aaddc6 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302087ea13cc3ad974405b316c6ade3aaddc6 2023-02-08 23:56:15,140 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302086535f1f4066e4f38945979e4fb4962f9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302086535f1f4066e4f38945979e4fb4962f9 2023-02-08 23:56:15,142 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/114aab595c794b61abe9933b9a2c60d9 2023-02-08 23:56:15,146 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/ddf1424cc68f42d8b6c8de25695ee44a as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/ddf1424cc68f42d8b6c8de25695ee44a 2023-02-08 23:56:15,148 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/43a818148c734b81bbb05bd11999a2ee 2023-02-08 23:56:15,152 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/4e7c811bbd654ecdb311d5f10fa3c262 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/4e7c811bbd654ecdb311d5f10fa3c262 2023-02-08 23:56:15,160 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302082bdca3fbe9ea4e48ab976cea8206b6d4 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202302082bdca3fbe9ea4e48ab976cea8206b6d4 2023-02-08 23:56:15,179 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/ddf1424cc68f42d8b6c8de25695ee44a, entries=4, sequenceid=13, filesize=5.5 K 2023-02-08 23:56:15,179 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/114aab595c794b61abe9933b9a2c60d9 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/114aab595c794b61abe9933b9a2c60d9 2023-02-08 23:56:15,183 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/114aab595c794b61abe9933b9a2c60d9, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 23:56:15,184 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a3bc650bd8f95f4b16a947f7e690d376 in 567ms, sequenceid=13, compaction requested=true 2023-02-08 23:56:15,184 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for a3bc650bd8f95f4b16a947f7e690d376: 2023-02-08 23:56:15,184 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:15,184 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:56:15,188 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/4e7c811bbd654ecdb311d5f10fa3c262, entries=4, sequenceid=13, filesize=5.5 K 2023-02-08 23:56:15,189 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/43a818148c734b81bbb05bd11999a2ee as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/43a818148c734b81bbb05bd11999a2ee 2023-02-08 23:56:15,192 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/43a818148c734b81bbb05bd11999a2ee, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 23:56:15,193 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f0c73908a5b8d0551609b98f0efe640a in 576ms, sequenceid=13, compaction requested=true 2023-02-08 23:56:15,193 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0c73908a5b8d0551609b98f0efe640a: 2023-02-08 23:56:15,193 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:15,193 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:56:15,441 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 23:56:15,569 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/7efe432419b64b999547adbb18d17fbb 2023-02-08 23:56:15,574 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/ed3d16f9dceb41f2bc0c5c725c03978b as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/ed3d16f9dceb41f2bc0c5c725c03978b 2023-02-08 23:56:15,610 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/ed3d16f9dceb41f2bc0c5c725c03978b, entries=4, sequenceid=13, filesize=5.5 K 2023-02-08 23:56:15,611 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/7efe432419b64b999547adbb18d17fbb as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/7efe432419b64b999547adbb18d17fbb 2023-02-08 23:56:15,615 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/7efe432419b64b999547adbb18d17fbb, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 23:56:15,615 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for ab31217c6a226641e72afe8624c897d8 in 999ms, sequenceid=13, compaction requested=true 2023-02-08 23:56:15,616 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ab31217c6a226641e72afe8624c897d8: 2023-02-08 23:56:15,616 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:15,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:56:15,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:56:15,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 23:56:15,616 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 23:56:15,619 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,619 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:15,619 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:15,619 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:15,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:15,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:15,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:15,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 23:56:15,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@383f908d[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:15,621 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 23:56:15,621 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:15,622 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:15,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:15,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:15,622 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 23:56:15,622 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 23:56:15,622 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:15,622 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,622 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:15,623 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 23:56:15,623 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,623 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:15,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,623 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 23:56:15,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:15,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:15,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 23:56:15,626 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 23:56:15,626 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:15,626 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:15,626 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 23:56:15,626 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:15,627 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:15,627 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:15,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:15,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:15,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:15,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:15,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:15,627 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:15,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:15,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:15,628 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,629 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,632 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,632 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:15,632 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:15,632 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:15,632 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:15,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:15,632 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 23:56:15,632 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:15,632 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:15,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:15,632 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,632 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:15,632 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:15,632 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:15,633 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:15,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:15,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:15,633 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:15,733 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:15,733 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:15,733 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:15,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:15,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:15,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:15,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 23:56:15,737 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:15,737 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:15,738 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:15,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:15,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 23:56:15,740 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 23:56:15,740 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:15,740 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 23:56:15,740 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 23:56:15,740 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:15,740 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:15,744 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,744 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:15,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:15,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:15,744 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:15,744 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:15,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:15,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:15,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:15,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:15,745 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 23:56:15,745 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-08 23:56:15,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 23:56:15,745 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 23:56:15,746 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:15,746 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 23:56:15,746 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:15,746 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. started... 2023-02-08 23:56:15,746 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:56:15,746 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:15,746 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0c73908a5b8d0551609b98f0efe640a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 23:56:15,746 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:15,746 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:15,746 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. started... 2023-02-08 23:56:15,746 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. started... 2023-02-08 23:56:15,746 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing a3bc650bd8f95f4b16a947f7e690d376 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 23:56:15,746 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ab31217c6a226641e72afe8624c897d8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 23:56:15,803 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020865a91fabde2f4d50807a3c3d902f8c4e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020865a91fabde2f4d50807a3c3d902f8c4e 2023-02-08 23:56:16,204 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302088a3760b453e34d16a1e9f0aef0fb24ed to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302088a3760b453e34d16a1e9f0aef0fb24ed 2023-02-08 23:56:16,204 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208210fa28922074259bc701dd9d1b7f631 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208210fa28922074259bc701dd9d1b7f631 2023-02-08 23:56:16,219 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/22e348b0b2b54f6f9fb6dd40631b37fe 2023-02-08 23:56:16,220 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/c45dcf1c6a2f4733badb69408dec5eb0 2023-02-08 23:56:16,225 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/947a4983598e4a928cc67415754a3354 2023-02-08 23:56:16,257 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020839176f1604df4193a5f8c68a91e2a460 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023020839176f1604df4193a5f8c68a91e2a460 2023-02-08 23:56:16,260 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208abbbfa3c10f84b8b9db9ffd6a12db264 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230208abbbfa3c10f84b8b9db9ffd6a12db264 2023-02-08 23:56:16,260 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302085cfbab1bb48c4a958a43019fd5e269f9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302085cfbab1bb48c4a958a43019fd5e269f9 2023-02-08 23:56:16,265 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/1b544af4bd954376b83e9c792bc589a4 2023-02-08 23:56:16,268 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/22e348b0b2b54f6f9fb6dd40631b37fe as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/22e348b0b2b54f6f9fb6dd40631b37fe 2023-02-08 23:56:16,292 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/22e348b0b2b54f6f9fb6dd40631b37fe, entries=4, sequenceid=17, filesize=5.5 K 2023-02-08 23:56:16,292 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/1b544af4bd954376b83e9c792bc589a4 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/1b544af4bd954376b83e9c792bc589a4 2023-02-08 23:56:16,296 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/1b544af4bd954376b83e9c792bc589a4, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 23:56:16,297 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a3bc650bd8f95f4b16a947f7e690d376 in 551ms, sequenceid=17, compaction requested=true 2023-02-08 23:56:16,297 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for a3bc650bd8f95f4b16a947f7e690d376: 2023-02-08 23:56:16,297 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:16,297 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:56:16,662 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/e71417c991074c79a3ca66d0ab0ee2eb 2023-02-08 23:56:16,664 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/1ed5fbdb872c461c93d98ecbd7ecefa3 2023-02-08 23:56:16,666 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/947a4983598e4a928cc67415754a3354 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/947a4983598e4a928cc67415754a3354 2023-02-08 23:56:16,669 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/c45dcf1c6a2f4733badb69408dec5eb0 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/c45dcf1c6a2f4733badb69408dec5eb0 2023-02-08 23:56:16,702 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/c45dcf1c6a2f4733badb69408dec5eb0, entries=4, sequenceid=17, filesize=5.5 K 2023-02-08 23:56:16,702 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/947a4983598e4a928cc67415754a3354, entries=4, sequenceid=17, filesize=5.5 K 2023-02-08 23:56:16,703 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/1ed5fbdb872c461c93d98ecbd7ecefa3 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/1ed5fbdb872c461c93d98ecbd7ecefa3 2023-02-08 23:56:16,703 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/e71417c991074c79a3ca66d0ab0ee2eb as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/e71417c991074c79a3ca66d0ab0ee2eb 2023-02-08 23:56:16,707 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/1ed5fbdb872c461c93d98ecbd7ecefa3, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 23:56:16,707 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/e71417c991074c79a3ca66d0ab0ee2eb, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 23:56:16,708 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for ab31217c6a226641e72afe8624c897d8 in 962ms, sequenceid=17, compaction requested=true 2023-02-08 23:56:16,708 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f0c73908a5b8d0551609b98f0efe640a in 962ms, sequenceid=17, compaction requested=true 2023-02-08 23:56:16,708 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ab31217c6a226641e72afe8624c897d8: 2023-02-08 23:56:16,708 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0c73908a5b8d0551609b98f0efe640a: 2023-02-08 23:56:16,708 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:16,708 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:16,709 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:56:16,709 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:56:16,709 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:56:16,709 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,709 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 23:56:16,709 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 23:56:16,710 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:16,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:16,710 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:16,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:16,711 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:16,711 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:16,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:16,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:16,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:16,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 23:56:16,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fbba703[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:16,712 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 23:56:16,712 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:16,714 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:16,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:16,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:16,714 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 23:56:16,714 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 23:56:16,714 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:16,714 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,714 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:16,716 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 23:56:16,716 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,716 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:16,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:16,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:16,716 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 23:56:16,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:16,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:16,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:16,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:16,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:16,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:16,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 23:56:16,718 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 23:56:16,718 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:16,718 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:16,718 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 23:56:16,718 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:16,720 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:16,720 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:16,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:16,720 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:16,720 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,721 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:16,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:16,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,723 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,723 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:16,723 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:16,723 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:16,723 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 23:56:16,723 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:16,723 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:16,723 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:16,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:16,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:16,723 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,724 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:16,724 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:16,724 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:16,724 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:16,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:16,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:16,724 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:16,824 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:16,824 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:16,824 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:16,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:16,834 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:56:16,840 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:56:16,847 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:56:16,848 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:56:16,932 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:16,932 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:16,932 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 23:56:16,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-08 23:56:16,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 23:56:16,934 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 23:56:16,934 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:16,934 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 23:56:16,934 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 23:56:16,935 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:16,935 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:16,937 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:16,937 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:16,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:16,937 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:16,937 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 23:56:16,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:16,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:16,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 23:56:16,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:16,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 23:56:16,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-08 23:56:16,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 23:56:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 23:56:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 23:56:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 23:56:16,938 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:16,938 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. started... 2023-02-08 23:56:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 23:56:16,938 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:16,939 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f0c73908a5b8d0551609b98f0efe640a 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 23:56:16,938 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:16,938 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 23:56:16,939 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. started... 2023-02-08 23:56:16,939 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. started... 2023-02-08 23:56:16,939 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ab31217c6a226641e72afe8624c897d8 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 23:56:16,939 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing a3bc650bd8f95f4b16a947f7e690d376 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 23:56:17,776 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/6847e27bff52493ca7e6bb25d8d4bd81 2023-02-08 23:56:17,779 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/9751ddbf392f4a0d92879b50afe2d2b4 2023-02-08 23:56:17,779 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/b98da0b71f494b1e8298eebd8eb003f7 2023-02-08 23:56:17,807 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6847e27bff52493ca7e6bb25d8d4bd81 2023-02-08 23:56:17,809 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b98da0b71f494b1e8298eebd8eb003f7 2023-02-08 23:56:17,814 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9751ddbf392f4a0d92879b50afe2d2b4 2023-02-08 23:56:18,227 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/70f1b11c293c49db949c2a30dcdd218d 2023-02-08 23:56:18,230 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70f1b11c293c49db949c2a30dcdd218d 2023-02-08 23:56:18,231 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/b98da0b71f494b1e8298eebd8eb003f7 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/b98da0b71f494b1e8298eebd8eb003f7 2023-02-08 23:56:18,234 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/78cbf3dd88414c908ffd0924a59b0a36 2023-02-08 23:56:18,237 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78cbf3dd88414c908ffd0924a59b0a36 2023-02-08 23:56:18,238 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/9751ddbf392f4a0d92879b50afe2d2b4 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/9751ddbf392f4a0d92879b50afe2d2b4 2023-02-08 23:56:18,256 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b98da0b71f494b1e8298eebd8eb003f7 2023-02-08 23:56:18,256 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/b98da0b71f494b1e8298eebd8eb003f7, entries=3, sequenceid=23, filesize=5.2 K 2023-02-08 23:56:18,257 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/70f1b11c293c49db949c2a30dcdd218d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/70f1b11c293c49db949c2a30dcdd218d 2023-02-08 23:56:18,260 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9751ddbf392f4a0d92879b50afe2d2b4 2023-02-08 23:56:18,260 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/9751ddbf392f4a0d92879b50afe2d2b4, entries=3, sequenceid=23, filesize=5.2 K 2023-02-08 23:56:18,261 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/78cbf3dd88414c908ffd0924a59b0a36 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/78cbf3dd88414c908ffd0924a59b0a36 2023-02-08 23:56:18,261 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 70f1b11c293c49db949c2a30dcdd218d 2023-02-08 23:56:18,262 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/70f1b11c293c49db949c2a30dcdd218d, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 23:56:18,262 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a3bc650bd8f95f4b16a947f7e690d376 in 1323ms, sequenceid=23, compaction requested=true 2023-02-08 23:56:18,262 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for a3bc650bd8f95f4b16a947f7e690d376: 2023-02-08 23:56:18,262 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:18,263 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 23:56:18,265 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 78cbf3dd88414c908ffd0924a59b0a36 2023-02-08 23:56:18,265 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/78cbf3dd88414c908ffd0924a59b0a36, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 23:56:18,265 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for ab31217c6a226641e72afe8624c897d8 in 1326ms, sequenceid=23, compaction requested=true 2023-02-08 23:56:18,266 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ab31217c6a226641e72afe8624c897d8: 2023-02-08 23:56:18,266 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:18,266 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 23:56:18,622 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/338330a81619488296d0722a0299526b 2023-02-08 23:56:18,626 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 338330a81619488296d0722a0299526b 2023-02-08 23:56:18,627 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/6847e27bff52493ca7e6bb25d8d4bd81 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/6847e27bff52493ca7e6bb25d8d4bd81 2023-02-08 23:56:18,649 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6847e27bff52493ca7e6bb25d8d4bd81 2023-02-08 23:56:18,650 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/6847e27bff52493ca7e6bb25d8d4bd81, entries=3, sequenceid=23, filesize=5.2 K 2023-02-08 23:56:18,650 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/338330a81619488296d0722a0299526b as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/338330a81619488296d0722a0299526b 2023-02-08 23:56:18,653 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 338330a81619488296d0722a0299526b 2023-02-08 23:56:18,653 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/338330a81619488296d0722a0299526b, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 23:56:18,654 INFO [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f0c73908a5b8d0551609b98f0efe640a in 1716ms, sequenceid=23, compaction requested=true 2023-02-08 23:56:18,654 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f0c73908a5b8d0551609b98f0efe640a: 2023-02-08 23:56:18,654 DEBUG [rs(jenkins-hbase4.apache.org,42703,1675900414768)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:18,654 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 23:56:18,654 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 23:56:18,654 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,654 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 23:56:18,654 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 23:56:18,656 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,656 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:18,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:18,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:18,656 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:18,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:18,656 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 23:56:18,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:18,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:18,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:18,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42703,1675900414768' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 23:56:18,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6671f9b9[Count = 0] remaining members to acquire global barrier 2023-02-08 23:56:18,657 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 23:56:18,657 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:18,659 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:18,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:18,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:18,659 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 23:56:18,659 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 23:56:18,659 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42703,1675900414768' in zk 2023-02-08 23:56:18,659 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,659 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 23:56:18,660 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 23:56:18,660 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:18,660 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,660 DEBUG [member: 'jenkins-hbase4.apache.org,42703,1675900414768' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 23:56:18,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:18,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:18,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:18,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:18,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:18,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:18,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:18,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42703,1675900414768': 2023-02-08 23:56:18,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42703,1675900414768' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 23:56:18,662 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 23:56:18,662 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 23:56:18,662 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 23:56:18,662 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 23:56:18,662 INFO [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 23:56:18,665 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:18,665 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:18,665 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 23:56:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 23:56:18,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 23:56:18,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:18,666 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:18,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:18,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 23:56:18,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:18,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,666 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 23:56:18,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 23:56:18,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:18,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 23:56:18,670 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 23:56:18,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 23:56:18,670 DEBUG [(jenkins-hbase4.apache.org,39489,1675900413627)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:18,670 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 23:56:18,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 23:56:18,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 23:56:18,670 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 23:56:18,671 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 23:56:18,771 DEBUG [Listener at localhost/41521] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 23:56:18,771 INFO [Listener at localhost/41521] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:18,771 INFO [Listener at localhost/41521] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:18,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39489] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 23:56:18,772 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:56:18,772 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): f0c73908a5b8d0551609b98f0efe640a/family1 is initiating major compaction (all files) 2023-02-08 23:56:18,772 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:56:18,772 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:56:18,774 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of f0c73908a5b8d0551609b98f0efe640a/family1 in testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:18,774 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/1ec1500eb2d5405493775cf50ca4301c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/e8f71c67b2cd45f5bda4b6a3c26643e2, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/4e7c811bbd654ecdb311d5f10fa3c262, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/947a4983598e4a928cc67415754a3354, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/6847e27bff52493ca7e6bb25d8d4bd81] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp, totalSize=27.1 K 2023-02-08 23:56:18,775 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 1ec1500eb2d5405493775cf50ca4301c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900570868 2023-02-08 23:56:18,775 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting e8f71c67b2cd45f5bda4b6a3c26643e2, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900572740 2023-02-08 23:56:18,776 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 4e7c811bbd654ecdb311d5f10fa3c262, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900574608 2023-02-08 23:56:18,776 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 947a4983598e4a928cc67415754a3354, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900575735 2023-02-08 23:56:18,776 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 6847e27bff52493ca7e6bb25d8d4bd81, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:56:18,785 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): f0c73908a5b8d0551609b98f0efe640a#family1#compaction#197 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:56:19,222 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230208cc27cfd72e8544c8a867d497441bb8f2_del 2023-02-08 23:56:19,222 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208cc27cfd72e8544c8a867d497441bb8f2_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208cc27cfd72e8544c8a867d497441bb8f2_del 2023-02-08 23:56:20,084 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17f030e4ecff4458a5622e25f341c09d 2023-02-08 23:56:20,084 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family1/17f030e4ecff4458a5622e25f341c09d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/17f030e4ecff4458a5622e25f341c09d 2023-02-08 23:56:20,107 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17f030e4ecff4458a5622e25f341c09d 2023-02-08 23:56:20,108 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f0c73908a5b8d0551609b98f0efe640a/family1 of f0c73908a5b8d0551609b98f0efe640a into 17f030e4ecff4458a5622e25f341c09d(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:56:20,108 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for f0c73908a5b8d0551609b98f0efe640a: 2023-02-08 23:56:20,108 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:56:20,108 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): f0c73908a5b8d0551609b98f0efe640a/family2 is initiating major compaction (all files) 2023-02-08 23:56:20,109 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:56:20,109 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:56:20,110 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of f0c73908a5b8d0551609b98f0efe640a/family2 in testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:20,110 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/a3b3ca04283648629418ab49ec9e6dff, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/ba34eebffdd946948e37bd26d1e95717, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/43a818148c734b81bbb05bd11999a2ee, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/e71417c991074c79a3ca66d0ab0ee2eb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/338330a81619488296d0722a0299526b] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp, totalSize=25.4 K 2023-02-08 23:56:20,110 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting a3b3ca04283648629418ab49ec9e6dff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900570868 2023-02-08 23:56:20,111 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting ba34eebffdd946948e37bd26d1e95717, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900572740 2023-02-08 23:56:20,111 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 43a818148c734b81bbb05bd11999a2ee, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900574608 2023-02-08 23:56:20,111 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting e71417c991074c79a3ca66d0ab0ee2eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900575735 2023-02-08 23:56:20,112 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 338330a81619488296d0722a0299526b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:56:20,120 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): f0c73908a5b8d0551609b98f0efe640a#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:56:20,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/1ec1500eb2d5405493775cf50ca4301c, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/e8f71c67b2cd45f5bda4b6a3c26643e2, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/4e7c811bbd654ecdb311d5f10fa3c262, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/947a4983598e4a928cc67415754a3354, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/6847e27bff52493ca7e6bb25d8d4bd81] to archive 2023-02-08 23:56:20,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:20,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/1ec1500eb2d5405493775cf50ca4301c to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/1ec1500eb2d5405493775cf50ca4301c 2023-02-08 23:56:20,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/e8f71c67b2cd45f5bda4b6a3c26643e2 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/e8f71c67b2cd45f5bda4b6a3c26643e2 2023-02-08 23:56:20,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/4e7c811bbd654ecdb311d5f10fa3c262 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/4e7c811bbd654ecdb311d5f10fa3c262 2023-02-08 23:56:20,131 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/947a4983598e4a928cc67415754a3354 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/947a4983598e4a928cc67415754a3354 2023-02-08 23:56:20,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/6847e27bff52493ca7e6bb25d8d4bd81 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/6847e27bff52493ca7e6bb25d8d4bd81 2023-02-08 23:56:20,531 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302083e134e2080724e39bd93ed998452a724_del 2023-02-08 23:56:20,531 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302083e134e2080724e39bd93ed998452a724_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302083e134e2080724e39bd93ed998452a724_del 2023-02-08 23:56:21,343 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b55292b1d7a47919893c23c2b07c534 2023-02-08 23:56:21,344 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/.tmp/family2/4b55292b1d7a47919893c23c2b07c534 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/4b55292b1d7a47919893c23c2b07c534 2023-02-08 23:56:21,347 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b55292b1d7a47919893c23c2b07c534 2023-02-08 23:56:21,348 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f0c73908a5b8d0551609b98f0efe640a/family2 of f0c73908a5b8d0551609b98f0efe640a into 4b55292b1d7a47919893c23c2b07c534(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:56:21,348 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for f0c73908a5b8d0551609b98f0efe640a: 2023-02-08 23:56:21,349 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:56:21,349 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): a3bc650bd8f95f4b16a947f7e690d376/family1 is initiating major compaction (all files) 2023-02-08 23:56:21,349 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:56:21,349 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:56:21,350 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of a3bc650bd8f95f4b16a947f7e690d376/family1 in testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:21,350 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/0165b15c056a4b14bc787da762d2d9f5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/3bfa9a7a94294c489506ec21de5d54f4, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/ddf1424cc68f42d8b6c8de25695ee44a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/22e348b0b2b54f6f9fb6dd40631b37fe, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/b98da0b71f494b1e8298eebd8eb003f7] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp, totalSize=27.1 K 2023-02-08 23:56:21,350 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 0165b15c056a4b14bc787da762d2d9f5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900570867 2023-02-08 23:56:21,350 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 3bfa9a7a94294c489506ec21de5d54f4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900572740 2023-02-08 23:56:21,351 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting ddf1424cc68f42d8b6c8de25695ee44a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900574607 2023-02-08 23:56:21,351 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 22e348b0b2b54f6f9fb6dd40631b37fe, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900575735 2023-02-08 23:56:21,352 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting b98da0b71f494b1e8298eebd8eb003f7, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:56:21,360 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): a3bc650bd8f95f4b16a947f7e690d376#family1#compaction#199 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:56:21,406 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230208c88a6159181042d4bafc297bbb338cc0_del 2023-02-08 23:56:21,407 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208c88a6159181042d4bafc297bbb338cc0_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208c88a6159181042d4bafc297bbb338cc0_del 2023-02-08 23:56:21,424 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/a3b3ca04283648629418ab49ec9e6dff, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/ba34eebffdd946948e37bd26d1e95717, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/43a818148c734b81bbb05bd11999a2ee, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/e71417c991074c79a3ca66d0ab0ee2eb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/338330a81619488296d0722a0299526b] to archive 2023-02-08 23:56:21,424 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:21,426 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/a3b3ca04283648629418ab49ec9e6dff to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/a3b3ca04283648629418ab49ec9e6dff 2023-02-08 23:56:21,427 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/ba34eebffdd946948e37bd26d1e95717 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/ba34eebffdd946948e37bd26d1e95717 2023-02-08 23:56:21,429 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/43a818148c734b81bbb05bd11999a2ee to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/43a818148c734b81bbb05bd11999a2ee 2023-02-08 23:56:21,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/e71417c991074c79a3ca66d0ab0ee2eb to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/e71417c991074c79a3ca66d0ab0ee2eb 2023-02-08 23:56:21,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/338330a81619488296d0722a0299526b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family2/338330a81619488296d0722a0299526b 2023-02-08 23:56:21,861 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 847ed04b244c401dad23e3dd75c0bce6 2023-02-08 23:56:21,862 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family1/847ed04b244c401dad23e3dd75c0bce6 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/847ed04b244c401dad23e3dd75c0bce6 2023-02-08 23:56:21,884 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 847ed04b244c401dad23e3dd75c0bce6 2023-02-08 23:56:21,885 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a3bc650bd8f95f4b16a947f7e690d376/family1 of a3bc650bd8f95f4b16a947f7e690d376 into 847ed04b244c401dad23e3dd75c0bce6(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:56:21,885 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for a3bc650bd8f95f4b16a947f7e690d376: 2023-02-08 23:56:21,885 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:56:21,885 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): a3bc650bd8f95f4b16a947f7e690d376/family2 is initiating major compaction (all files) 2023-02-08 23:56:21,886 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:56:21,886 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:56:21,886 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of a3bc650bd8f95f4b16a947f7e690d376/family2 in testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:21,887 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/71fb3bd8f6f147f2a9b1e509b0f780e0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/6f6ce1f80b1c4570a18c0b905671ef42, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/114aab595c794b61abe9933b9a2c60d9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/1b544af4bd954376b83e9c792bc589a4, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/70f1b11c293c49db949c2a30dcdd218d] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp, totalSize=25.4 K 2023-02-08 23:56:21,887 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 71fb3bd8f6f147f2a9b1e509b0f780e0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900570867 2023-02-08 23:56:21,887 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 6f6ce1f80b1c4570a18c0b905671ef42, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900572740 2023-02-08 23:56:21,888 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 114aab595c794b61abe9933b9a2c60d9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900574607 2023-02-08 23:56:21,888 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 1b544af4bd954376b83e9c792bc589a4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900575735 2023-02-08 23:56:21,888 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 70f1b11c293c49db949c2a30dcdd218d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:56:21,896 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): a3bc650bd8f95f4b16a947f7e690d376#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:56:21,906 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302080787b6f0dce74177861558a80d0d94f3_del 2023-02-08 23:56:21,906 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302080787b6f0dce74177861558a80d0d94f3_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302080787b6f0dce74177861558a80d0d94f3_del 2023-02-08 23:56:21,925 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/0165b15c056a4b14bc787da762d2d9f5, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/3bfa9a7a94294c489506ec21de5d54f4, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/ddf1424cc68f42d8b6c8de25695ee44a, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/22e348b0b2b54f6f9fb6dd40631b37fe, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/b98da0b71f494b1e8298eebd8eb003f7] to archive 2023-02-08 23:56:21,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:21,927 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/0165b15c056a4b14bc787da762d2d9f5 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/0165b15c056a4b14bc787da762d2d9f5 2023-02-08 23:56:21,928 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/3bfa9a7a94294c489506ec21de5d54f4 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/3bfa9a7a94294c489506ec21de5d54f4 2023-02-08 23:56:21,929 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/ddf1424cc68f42d8b6c8de25695ee44a to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/ddf1424cc68f42d8b6c8de25695ee44a 2023-02-08 23:56:21,930 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/22e348b0b2b54f6f9fb6dd40631b37fe to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/22e348b0b2b54f6f9fb6dd40631b37fe 2023-02-08 23:56:21,931 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/b98da0b71f494b1e8298eebd8eb003f7 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/b98da0b71f494b1e8298eebd8eb003f7 2023-02-08 23:56:22,719 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea3a361b23374a65a4a985e3ffdc5707 2023-02-08 23:56:22,720 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/.tmp/family2/ea3a361b23374a65a4a985e3ffdc5707 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/ea3a361b23374a65a4a985e3ffdc5707 2023-02-08 23:56:22,724 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea3a361b23374a65a4a985e3ffdc5707 2023-02-08 23:56:22,725 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a3bc650bd8f95f4b16a947f7e690d376/family2 of a3bc650bd8f95f4b16a947f7e690d376 into ea3a361b23374a65a4a985e3ffdc5707(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:56:22,725 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for a3bc650bd8f95f4b16a947f7e690d376: 2023-02-08 23:56:22,725 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:56:22,725 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): ab31217c6a226641e72afe8624c897d8/family1 is initiating major compaction (all files) 2023-02-08 23:56:22,725 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:56:22,726 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:56:22,726 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of ab31217c6a226641e72afe8624c897d8/family1 in testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:22,727 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/6be73641c36f4217bf35767e4961c8e3, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/65cdfb10f8dc4cc695523e80c46c049d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/ed3d16f9dceb41f2bc0c5c725c03978b, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/c45dcf1c6a2f4733badb69408dec5eb0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/9751ddbf392f4a0d92879b50afe2d2b4] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp, totalSize=27.1 K 2023-02-08 23:56:22,727 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 6be73641c36f4217bf35767e4961c8e3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900570868 2023-02-08 23:56:22,727 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 65cdfb10f8dc4cc695523e80c46c049d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900572740 2023-02-08 23:56:22,728 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting ed3d16f9dceb41f2bc0c5c725c03978b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900574608 2023-02-08 23:56:22,728 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting c45dcf1c6a2f4733badb69408dec5eb0, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900575735 2023-02-08 23:56:22,728 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 9751ddbf392f4a0d92879b50afe2d2b4, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:56:22,737 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): ab31217c6a226641e72afe8624c897d8#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:56:22,824 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/71fb3bd8f6f147f2a9b1e509b0f780e0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/6f6ce1f80b1c4570a18c0b905671ef42, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/114aab595c794b61abe9933b9a2c60d9, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/1b544af4bd954376b83e9c792bc589a4, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/70f1b11c293c49db949c2a30dcdd218d] to archive 2023-02-08 23:56:22,825 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:22,826 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/71fb3bd8f6f147f2a9b1e509b0f780e0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/71fb3bd8f6f147f2a9b1e509b0f780e0 2023-02-08 23:56:22,827 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/6f6ce1f80b1c4570a18c0b905671ef42 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/6f6ce1f80b1c4570a18c0b905671ef42 2023-02-08 23:56:22,828 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/114aab595c794b61abe9933b9a2c60d9 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/114aab595c794b61abe9933b9a2c60d9 2023-02-08 23:56:22,829 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/1b544af4bd954376b83e9c792bc589a4 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/1b544af4bd954376b83e9c792bc589a4 2023-02-08 23:56:22,830 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/70f1b11c293c49db949c2a30dcdd218d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family2/70f1b11c293c49db949c2a30dcdd218d 2023-02-08 23:56:23,175 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302081b152fbd1dce45f296172fbe3104bd16_del 2023-02-08 23:56:23,175 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302081b152fbd1dce45f296172fbe3104bd16_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302081b152fbd1dce45f296172fbe3104bd16_del 2023-02-08 23:56:24,028 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2bb836f9db00463798d81f8679a401b7 2023-02-08 23:56:24,029 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family1/2bb836f9db00463798d81f8679a401b7 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/2bb836f9db00463798d81f8679a401b7 2023-02-08 23:56:24,060 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2bb836f9db00463798d81f8679a401b7 2023-02-08 23:56:24,061 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ab31217c6a226641e72afe8624c897d8/family1 of ab31217c6a226641e72afe8624c897d8 into 2bb836f9db00463798d81f8679a401b7(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 23:56:24,061 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for ab31217c6a226641e72afe8624c897d8: 2023-02-08 23:56:24,061 DEBUG [Listener at localhost/41521] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 23:56:24,061 DEBUG [Listener at localhost/41521] regionserver.HStore(1912): ab31217c6a226641e72afe8624c897d8/family2 is initiating major compaction (all files) 2023-02-08 23:56:24,061 INFO [Listener at localhost/41521] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 23:56:24,061 INFO [Listener at localhost/41521] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 23:56:24,061 INFO [Listener at localhost/41521] regionserver.HRegion(2258): Starting compaction of ab31217c6a226641e72afe8624c897d8/family2 in testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:24,061 INFO [Listener at localhost/41521] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/491515f6e7a44b7fa68e957d2bc0a295, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/e975b54b93eb4992876c6508ad8b2497, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/7efe432419b64b999547adbb18d17fbb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/1ed5fbdb872c461c93d98ecbd7ecefa3, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/78cbf3dd88414c908ffd0924a59b0a36] into tmpdir=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp, totalSize=25.4 K 2023-02-08 23:56:24,062 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 491515f6e7a44b7fa68e957d2bc0a295, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675900570868 2023-02-08 23:56:24,062 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting e975b54b93eb4992876c6508ad8b2497, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675900572740 2023-02-08 23:56:24,062 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 7efe432419b64b999547adbb18d17fbb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675900574608 2023-02-08 23:56:24,063 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 1ed5fbdb872c461c93d98ecbd7ecefa3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675900575735 2023-02-08 23:56:24,063 DEBUG [Listener at localhost/41521] compactions.Compactor(207): Compacting 78cbf3dd88414c908ffd0924a59b0a36, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 23:56:24,070 INFO [Listener at localhost/41521] throttle.PressureAwareThroughputController(145): ab31217c6a226641e72afe8624c897d8#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 23:56:24,076 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302080d1c82a257f343a789ff952df3ff5642_del 2023-02-08 23:56:24,076 INFO [Listener at localhost/41521] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302080d1c82a257f343a789ff952df3ff5642_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202302080d1c82a257f343a789ff952df3ff5642_del 2023-02-08 23:56:24,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/6be73641c36f4217bf35767e4961c8e3, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/65cdfb10f8dc4cc695523e80c46c049d, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/ed3d16f9dceb41f2bc0c5c725c03978b, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/c45dcf1c6a2f4733badb69408dec5eb0, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/9751ddbf392f4a0d92879b50afe2d2b4] to archive 2023-02-08 23:56:24,126 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:24,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/6be73641c36f4217bf35767e4961c8e3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/6be73641c36f4217bf35767e4961c8e3 2023-02-08 23:56:24,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/65cdfb10f8dc4cc695523e80c46c049d to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/65cdfb10f8dc4cc695523e80c46c049d 2023-02-08 23:56:24,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/ed3d16f9dceb41f2bc0c5c725c03978b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/ed3d16f9dceb41f2bc0c5c725c03978b 2023-02-08 23:56:24,130 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/c45dcf1c6a2f4733badb69408dec5eb0 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/c45dcf1c6a2f4733badb69408dec5eb0 2023-02-08 23:56:24,130 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/9751ddbf392f4a0d92879b50afe2d2b4 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/9751ddbf392f4a0d92879b50afe2d2b4 2023-02-08 23:56:24,889 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 98078d396ec34286acc3520982c6f544 2023-02-08 23:56:24,890 DEBUG [Listener at localhost/41521] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/.tmp/family2/98078d396ec34286acc3520982c6f544 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/98078d396ec34286acc3520982c6f544 2023-02-08 23:56:24,893 INFO [Listener at localhost/41521] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 98078d396ec34286acc3520982c6f544 2023-02-08 23:56:24,894 INFO [Listener at localhost/41521] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ab31217c6a226641e72afe8624c897d8/family2 of ab31217c6a226641e72afe8624c897d8 into 98078d396ec34286acc3520982c6f544(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 23:56:24,894 DEBUG [Listener at localhost/41521] regionserver.HRegion(2288): Compaction status journal for ab31217c6a226641e72afe8624c897d8: 2023-02-08 23:56:24,897 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:56:24,900 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:56:24,903 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:56:24,904 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:56:24,908 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39489] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCA*f{\xCF\x9E\xB3/W\xB2\xAF\x14\xEF\x18j\xBF"\x10\x9Fy\x8B~\xACP\xA0}\x84pVGZ\xCA\x989*\x10\x9C\x0A\xAC\xEBA\x0B\xA8\x0E\x1C\x04Zu`\xE2#\xD12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-08 23:56:24,929 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-08 23:56:24,930 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-08 23:56:24,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:56:24,930 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675901184930 2023-02-08 23:56:24,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 23:56:24,931 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/491515f6e7a44b7fa68e957d2bc0a295, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/e975b54b93eb4992876c6508ad8b2497, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/7efe432419b64b999547adbb18d17fbb, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/1ed5fbdb872c461c93d98ecbd7ecefa3, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/78cbf3dd88414c908ffd0924a59b0a36] to archive 2023-02-08 23:56:24,932 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:24,933 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/491515f6e7a44b7fa68e957d2bc0a295 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/491515f6e7a44b7fa68e957d2bc0a295 2023-02-08 23:56:24,934 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/e975b54b93eb4992876c6508ad8b2497 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/e975b54b93eb4992876c6508ad8b2497 2023-02-08 23:56:24,936 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/7efe432419b64b999547adbb18d17fbb to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/7efe432419b64b999547adbb18d17fbb 2023-02-08 23:56:24,937 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/1ed5fbdb872c461c93d98ecbd7ecefa3 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/1ed5fbdb872c461c93d98ecbd7ecefa3 2023-02-08 23:56:24,937 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/78cbf3dd88414c908ffd0924a59b0a36 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family2/78cbf3dd88414c908ffd0924a59b0a36 2023-02-08 23:56:25,260 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-08 23:56:25,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230208c88a6159181042d4bafc297bbb338cc0_del 2023-02-08 23:56:25,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302081b152fbd1dce45f296172fbe3104bd16_del 2023-02-08 23:56:25,324 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230208cc27cfd72e8544c8a867d497441bb8f2_del 2023-02-08 23:56:25,324 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 23:56:25,325 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] zookeeper.ReadOnlyZKClient(139): Connect 0x591e3142 to 127.0.0.1:54774 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 23:56:25,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ff9b799, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 23:56:25,330 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 23:56:25,389 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 23:56:25,877 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020838be44ebf4024b0f9261d4785b86e900 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020838be44ebf4024b0f9261d4785b86e900 2023-02-08 23:56:25,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 23:56:25,888 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 23:56:25,891 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55454, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 23:56:25,898 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:25,898 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:25,900 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 23:56:25,902 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49170, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 23:56:25,942 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208/family1/90ea2f54b6db4807a3c7c9770dbb5466 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-08 23:56:25,943 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208/family1/90ea2f54b6db4807a3c7c9770dbb5466}] 2023-02-08 23:56:25,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208/family1/90ea2f54b6db4807a3c7c9770dbb5466 for inclusion in a3bc650bd8f95f4b16a947f7e690d376/family1 2023-02-08 23:56:25,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-08 23:56:25,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first= last=1 2023-02-08 23:56:25,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for a3bc650bd8f95f4b16a947f7e690d376: 2023-02-08 23:56:25,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208/family1/90ea2f54b6db4807a3c7c9770dbb5466 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__n406go0bipit54clptir9l732uen7vohidlopenioa80i3biq6ppvt3o5cm5kj41/family1/90ea2f54b6db4807a3c7c9770dbb5466 2023-02-08 23:56:25,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__n406go0bipit54clptir9l732uen7vohidlopenioa80i3biq6ppvt3o5cm5kj41/family1/90ea2f54b6db4807a3c7c9770dbb5466 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/d13527b9b68a41b19942a8a667debb4a_SeqId_28_ 2023-02-08 23:56:25,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__n406go0bipit54clptir9l732uen7vohidlopenioa80i3biq6ppvt3o5cm5kj41/family1/90ea2f54b6db4807a3c7c9770dbb5466 into a3bc650bd8f95f4b16a947f7e690d376/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/d13527b9b68a41b19942a8a667debb4a_SeqId_28_ - updating store file list. 2023-02-08 23:56:25,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/d13527b9b68a41b19942a8a667debb4a_SeqId_28_ into a3bc650bd8f95f4b16a947f7e690d376/family1 2023-02-08 23:56:25,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__n406go0bipit54clptir9l732uen7vohidlopenioa80i3biq6ppvt3o5cm5kj41/family1/90ea2f54b6db4807a3c7c9770dbb5466 into a3bc650bd8f95f4b16a947f7e690d376/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/family1/d13527b9b68a41b19942a8a667debb4a_SeqId_28_) 2023-02-08 23:56:25,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__n406go0bipit54clptir9l732uen7vohidlopenioa80i3biq6ppvt3o5cm5kj41/family1/90ea2f54b6db4807a3c7c9770dbb5466 2023-02-08 23:56:26,001 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 23:56:26,001 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:26,003 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020821e5d8cb14e54df2b1b630b4f32ef29b to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023020821e5d8cb14e54df2b1b630b4f32ef29b 2023-02-08 23:56:26,004 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302085ee50b3aac824da19d41887360469353 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302085ee50b3aac824da19d41887360469353 2023-02-08 23:56:26,004 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302088a3760b453e34d16a1e9f0aef0fb24ed to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302088a3760b453e34d16a1e9f0aef0fb24ed 2023-02-08 23:56:26,005 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208add635f9e8c24723997118e4a0974d71 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208add635f9e8c24723997118e4a0974d71 2023-02-08 23:56:26,005 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-08 23:56:26,005 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 23:56:26,006 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 23:56:26,006 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 23:56:26,580 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208eb381f964a394521a3608dd70196a3db to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208eb381f964a394521a3608dd70196a3db 2023-02-08 23:56:26,581 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 23:56:26,601 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:26,601 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:26,629 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 23:56:26,635 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208/family1/7f01ffb77b014f39831e420d269794af first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-08 23:56:26,636 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208/family1/7f01ffb77b014f39831e420d269794af}] 2023-02-08 23:56:26,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208/family1/7f01ffb77b014f39831e420d269794af for inclusion in ab31217c6a226641e72afe8624c897d8/family1 2023-02-08 23:56:26,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-08 23:56:26,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-08 23:56:26,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for ab31217c6a226641e72afe8624c897d8: 2023-02-08 23:56:26,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208/family1/7f01ffb77b014f39831e420d269794af to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__ft8md5sv4jcmo6pf92pt4ftt7ah24tkgqpukga90t37juq5occv2tb0dv4jancad/family1/7f01ffb77b014f39831e420d269794af 2023-02-08 23:56:26,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__ft8md5sv4jcmo6pf92pt4ftt7ah24tkgqpukga90t37juq5occv2tb0dv4jancad/family1/7f01ffb77b014f39831e420d269794af as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/90dbea92dce945099f3eadbb1c407245_SeqId_28_ 2023-02-08 23:56:26,664 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__ft8md5sv4jcmo6pf92pt4ftt7ah24tkgqpukga90t37juq5occv2tb0dv4jancad/family1/7f01ffb77b014f39831e420d269794af into ab31217c6a226641e72afe8624c897d8/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/90dbea92dce945099f3eadbb1c407245_SeqId_28_ - updating store file list. 2023-02-08 23:56:26,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/90dbea92dce945099f3eadbb1c407245_SeqId_28_ into ab31217c6a226641e72afe8624c897d8/family1 2023-02-08 23:56:26,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__ft8md5sv4jcmo6pf92pt4ftt7ah24tkgqpukga90t37juq5occv2tb0dv4jancad/family1/7f01ffb77b014f39831e420d269794af into ab31217c6a226641e72afe8624c897d8/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/family1/90dbea92dce945099f3eadbb1c407245_SeqId_28_) 2023-02-08 23:56:26,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__ft8md5sv4jcmo6pf92pt4ftt7ah24tkgqpukga90t37juq5occv2tb0dv4jancad/family1/7f01ffb77b014f39831e420d269794af 2023-02-08 23:56:26,691 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 23:56:26,692 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:26,693 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302081b706508c1004ed6a2d0a7e660aabc9f to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302081b706508c1004ed6a2d0a7e660aabc9f 2023-02-08 23:56:26,694 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208210fa28922074259bc701dd9d1b7f631 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208210fa28922074259bc701dd9d1b7f631 2023-02-08 23:56:26,695 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302087bf74f671c6a4969a0a9426c8d478e65 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302087bf74f671c6a4969a0a9426c8d478e65 2023-02-08 23:56:26,697 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208904593358edd4d0ca12671a4378861ac to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208904593358edd4d0ca12671a4378861ac 2023-02-08 23:56:26,697 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-08 23:56:26,697 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208' 2023-02-08 23:56:26,697 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 23:56:26,697 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 23:56:26,841 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:56:26,841 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e2f473c to 127.0.0.1:54774 2023-02-08 23:56:26,841 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:56:27,717 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020822eae26157304421b37b737d2daabd50 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020822eae26157304421b37b737d2daabd50 2023-02-08 23:56:27,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 23:56:27,733 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 23:56:27,733 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 23:56:27,759 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208/family1/04bf1929669d49228ec40c450b300818 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-08 23:56:27,759 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a., hostname=jenkins-hbase4.apache.org,42703,1675900414768, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208/family1/04bf1929669d49228ec40c450b300818}] 2023-02-08 23:56:27,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(763): Validating hfile at hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208/family1/04bf1929669d49228ec40c450b300818 for inclusion in f0c73908a5b8d0551609b98f0efe640a/family1 2023-02-08 23:56:27,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-08 23:56:27,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 23:56:27,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegion(2445): Flush status journal for f0c73908a5b8d0551609b98f0efe640a: 2023-02-08 23:56:27,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208/family1/04bf1929669d49228ec40c450b300818 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__uc7ogn7a3skq695cumdq27ne4d5m0vj3vomr0p98ai55bsk2umann7r0njs7qq2q/family1/04bf1929669d49228ec40c450b300818 2023-02-08 23:56:27,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__uc7ogn7a3skq695cumdq27ne4d5m0vj3vomr0p98ai55bsk2umann7r0njs7qq2q/family1/04bf1929669d49228ec40c450b300818 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/23eb4ca4384b4f31a6fcf86d94ff1f41_SeqId_28_ 2023-02-08 23:56:27,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(849): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__uc7ogn7a3skq695cumdq27ne4d5m0vj3vomr0p98ai55bsk2umann7r0njs7qq2q/family1/04bf1929669d49228ec40c450b300818 into f0c73908a5b8d0551609b98f0efe640a/family1 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/23eb4ca4384b4f31a6fcf86d94ff1f41_SeqId_28_ - updating store file list. 2023-02-08 23:56:27,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(882): Loaded HFile hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/23eb4ca4384b4f31a6fcf86d94ff1f41_SeqId_28_ into f0c73908a5b8d0551609b98f0efe640a/family1 2023-02-08 23:56:27,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.HStore(855): Successfully loaded hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__uc7ogn7a3skq695cumdq27ne4d5m0vj3vomr0p98ai55bsk2umann7r0njs7qq2q/family1/04bf1929669d49228ec40c450b300818 into f0c73908a5b8d0551609b98f0efe640a/family1 (new location: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/family1/23eb4ca4384b4f31a6fcf86d94ff1f41_SeqId_28_) 2023-02-08 23:56:27,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42703] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/staging/jenkins__testMajorCompactionFromAdmin__uc7ogn7a3skq695cumdq27ne4d5m0vj3vomr0p98ai55bsk2umann7r0njs7qq2q/family1/04bf1929669d49228ec40c450b300818 2023-02-08 23:56:27,809 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 23:56:27,810 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:27,811 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020865a91fabde2f4d50807a3c3d902f8c4e to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020865a91fabde2f4d50807a3c3d902f8c4e 2023-02-08 23:56:27,812 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302086aa839aaf90b4daea7cf2ebc2e605058 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302086aa839aaf90b4daea7cf2ebc2e605058 2023-02-08 23:56:27,813 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020894d6fb904b8a492ca70cf5dc00ef4989 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020894d6fb904b8a492ca70cf5dc00ef4989 2023-02-08 23:56:27,813 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208eb77a26a45894668ba30287dfd6b1c46 to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208eb77a26a45894668ba30287dfd6b1c46 2023-02-08 23:56:27,814 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-08 23:56:27,814 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 23:56:27,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 23:56:27,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 23:56:27,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 23:56:27,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(367): [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208c88a6159181042d4bafc297bbb338cc0_del] 2023-02-08 23:56:27,815 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:27,816 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208c88a6159181042d4bafc297bbb338cc0_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208c88a6159181042d4bafc297bbb338cc0_del 2023-02-08 23:56:27,816 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-08 23:56:27,816 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(367): [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302081b152fbd1dce45f296172fbe3104bd16_del] 2023-02-08 23:56:27,817 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:27,818 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302081b152fbd1dce45f296172fbe3104bd16_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302081b152fbd1dce45f296172fbe3104bd16_del 2023-02-08 23:56:27,818 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-08 23:56:27,818 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] compactions.PartitionedMobCompactor(367): [hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208cc27cfd72e8544c8a867d497441bb8f2_del] 2023-02-08 23:56:27,818 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 23:56:27,819 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208cc27cfd72e8544c8a867d497441bb8f2_del to hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208cc27cfd72e8544c8a867d497441bb8f2_del 2023-02-08 23:56:27,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-08 23:56:27,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675900418652] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 23:56:27,820 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-08 23:56:27,821 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.8920 sec 2023-02-08 23:56:27,839 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:56:27,841 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:56:27,843 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 23:56:27,843 DEBUG [Listener at localhost/41521] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 23:56:27,847 INFO [Listener at localhost/41521] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 23:56:27,881 INFO [Listener at localhost/41521] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 257), OpenFileDescriptor=642 (was 652), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=124 (was 100) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3271 (was 3283) 2023-02-08 23:56:27,881 DEBUG [Listener at localhost/41521] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33fc1e18 to 127.0.0.1:54774 2023-02-08 23:56:27,882 DEBUG [Listener at localhost/41521] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:56:27,882 INFO [Listener at localhost/41521] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-08 23:56:27,882 INFO [Listener at localhost/41521] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 23:56:27,882 DEBUG [Listener at localhost/41521] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0672b468 to 127.0.0.1:54774 2023-02-08 23:56:27,882 DEBUG [Listener at localhost/41521] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:56:27,882 DEBUG [Listener at localhost/41521] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-08 23:56:27,883 DEBUG [Listener at localhost/41521] util.JVMClusterUtil(257): Found active master hash=780862675, stopped=false 2023-02-08 23:56:27,883 INFO [Listener at localhost/41521] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:56:27,885 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 23:56:27,885 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 23:56:27,885 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:56:27,885 INFO [Listener at localhost/41521] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 23:56:27,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:56:27,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 23:56:27,885 DEBUG [Listener at localhost/41521] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0b955e48 to 127.0.0.1:54774 2023-02-08 23:56:27,885 DEBUG [Listener at localhost/41521] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:56:27,886 INFO [Listener at localhost/41521] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42703,1675900414768' ***** 2023-02-08 23:56:27,886 INFO [Listener at localhost/41521] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 23:56:27,886 INFO [RS:0;jenkins-hbase4:42703] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 23:56:27,886 INFO [RS:0;jenkins-hbase4:42703] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 23:56:27,886 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 23:56:27,886 INFO [RS:0;jenkins-hbase4:42703] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 23:56:27,887 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:27,887 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 9b83370f2c21546d39bae07b7ba35fa0 2023-02-08 23:56:27,887 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 31ce00ef22047cfd9f24d7e09ef0a684 2023-02-08 23:56:27,887 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for b22d2ff0c5597d74f67d046e7f16564e 2023-02-08 23:56:27,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f0c73908a5b8d0551609b98f0efe640a, disabling compactions & flushes 2023-02-08 23:56:27,887 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 539644099f8843cf0be59ce746f2f9e3 2023-02-08 23:56:27,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:27,887 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 893b8071d052c13d0194a6716bc74a8c 2023-02-08 23:56:27,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:27,887 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for a9e06470572e5280a9822eb7682ca46a 2023-02-08 23:56:27,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. after waiting 0 ms 2023-02-08 23:56:27,887 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 777160b56e576a2f3a7f98e3a2aa2080 2023-02-08 23:56:27,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 83e0134f0861e917b7a0acc88b520e50 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for a3bc650bd8f95f4b16a947f7e690d376 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 2f876068b9705192eb71b2d2aa908931 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 6ef0d93c124a8d762314a9d7b3e5e1da 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 857c72d9be3c2ff762195748d825aa71 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 9d98bce82dc93804dedabab1ead06ebc 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for ab31217c6a226641e72afe8624c897d8 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 9246c8eb0029ec8f06f1acd496f0cde7 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for bf85808cb96a3fd70743466476b776c5 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for c0432a17915eb080602d94742abd61bf 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 56b1dbb83cb5199116335c5260eadbb4 2023-02-08 23:56:27,888 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:27,888 DEBUG [RS:0;jenkins-hbase4:42703] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x01e7a4d7 to 127.0.0.1:54774 2023-02-08 23:56:27,888 DEBUG [RS:0;jenkins-hbase4:42703] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:56:27,889 INFO [RS:0;jenkins-hbase4:42703] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 23:56:27,889 INFO [RS:0;jenkins-hbase4:42703] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 23:56:27,889 INFO [RS:0;jenkins-hbase4:42703] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 23:56:27,889 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-08 23:56:27,889 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-08 23:56:27,889 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 23:56:27,889 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 23:56:27,889 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 23:56:27,889 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 23:56:27,889 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 23:56:27,889 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-02-08 23:56:27,890 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1505): Waiting on 1588230740, 2f876068b9705192eb71b2d2aa908931, 31ce00ef22047cfd9f24d7e09ef0a684, 539644099f8843cf0be59ce746f2f9e3, 56b1dbb83cb5199116335c5260eadbb4, 6ef0d93c124a8d762314a9d7b3e5e1da, 777160b56e576a2f3a7f98e3a2aa2080, 83e0134f0861e917b7a0acc88b520e50, 857c72d9be3c2ff762195748d825aa71, 893b8071d052c13d0194a6716bc74a8c, 9246c8eb0029ec8f06f1acd496f0cde7, 9b83370f2c21546d39bae07b7ba35fa0, 9d98bce82dc93804dedabab1ead06ebc, a3bc650bd8f95f4b16a947f7e690d376, a9e06470572e5280a9822eb7682ca46a, ab31217c6a226641e72afe8624c897d8, b22d2ff0c5597d74f67d046e7f16564e, bf85808cb96a3fd70743466476b776c5, c0432a17915eb080602d94742abd61bf, c091f7407da93dea8dbedd1c557bdac1, f0c73908a5b8d0551609b98f0efe640a 2023-02-08 23:56:27,892 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/f0c73908a5b8d0551609b98f0efe640a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 23:56:27,893 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:27,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f0c73908a5b8d0551609b98f0efe640a: Waiting for close lock at 1675900587887Running coprocessor pre-close hooks at 1675900587887Disabling compacts and flushes for region at 1675900587887Disabling writes for close at 1675900587887Writing region close event to WAL at 1675900587889 (+2 ms)Running coprocessor post-close hooks at 1675900587892 (+3 ms)Closed at 1675900587893 (+1 ms) 2023-02-08 23:56:27,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1675900568720.f0c73908a5b8d0551609b98f0efe640a. 2023-02-08 23:56:27,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9b83370f2c21546d39bae07b7ba35fa0, disabling compactions & flushes 2023-02-08 23:56:27,893 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:56:27,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:56:27,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. after waiting 0 ms 2023-02-08 23:56:27,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:56:27,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithMonthlyPolicy/9b83370f2c21546d39bae07b7ba35fa0/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-08 23:56:27,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:56:27,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9b83370f2c21546d39bae07b7ba35fa0: Waiting for close lock at 1675900587893Running coprocessor pre-close hooks at 1675900587893Disabling compacts and flushes for region at 1675900587893Disabling writes for close at 1675900587893Writing region close event to WAL at 1675900587894 (+1 ms)Running coprocessor post-close hooks at 1675900587901 (+7 ms)Closed at 1675900587902 (+1 ms) 2023-02-08 23:56:27,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1675900432221.9b83370f2c21546d39bae07b7ba35fa0. 2023-02-08 23:56:27,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 31ce00ef22047cfd9f24d7e09ef0a684, disabling compactions & flushes 2023-02-08 23:56:27,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:56:27,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:56:27,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. after waiting 0 ms 2023-02-08 23:56:27,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:56:27,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 31ce00ef22047cfd9f24d7e09ef0a684 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-08 23:56:27,959 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-08 23:56:27,959 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-08 23:56:27,959 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 23:56:27,959 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-08 23:56:27,969 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 23:56:28,090 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1505): Waiting on 1588230740, 2f876068b9705192eb71b2d2aa908931, 31ce00ef22047cfd9f24d7e09ef0a684, 539644099f8843cf0be59ce746f2f9e3, 56b1dbb83cb5199116335c5260eadbb4, 6ef0d93c124a8d762314a9d7b3e5e1da, 777160b56e576a2f3a7f98e3a2aa2080, 83e0134f0861e917b7a0acc88b520e50, 857c72d9be3c2ff762195748d825aa71, 893b8071d052c13d0194a6716bc74a8c, 9246c8eb0029ec8f06f1acd496f0cde7, 9d98bce82dc93804dedabab1ead06ebc, a3bc650bd8f95f4b16a947f7e690d376, a9e06470572e5280a9822eb7682ca46a, ab31217c6a226641e72afe8624c897d8, b22d2ff0c5597d74f67d046e7f16564e, bf85808cb96a3fd70743466476b776c5, c0432a17915eb080602d94742abd61bf, c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:56:28,290 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1505): Waiting on 1588230740, 2f876068b9705192eb71b2d2aa908931, 31ce00ef22047cfd9f24d7e09ef0a684, 539644099f8843cf0be59ce746f2f9e3, 56b1dbb83cb5199116335c5260eadbb4, 6ef0d93c124a8d762314a9d7b3e5e1da, 777160b56e576a2f3a7f98e3a2aa2080, 83e0134f0861e917b7a0acc88b520e50, 857c72d9be3c2ff762195748d825aa71, 893b8071d052c13d0194a6716bc74a8c, 9246c8eb0029ec8f06f1acd496f0cde7, 9d98bce82dc93804dedabab1ead06ebc, a3bc650bd8f95f4b16a947f7e690d376, a9e06470572e5280a9822eb7682ca46a, ab31217c6a226641e72afe8624c897d8, b22d2ff0c5597d74f67d046e7f16564e, bf85808cb96a3fd70743466476b776c5, c0432a17915eb080602d94742abd61bf, c091f7407da93dea8dbedd1c557bdac1 2023-02-08 23:56:28,303 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/.tmp/info/fe24e381d334492d9fa826c38e9e9201 2023-02-08 23:56:28,306 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fe24e381d334492d9fa826c38e9e9201 2023-02-08 23:56:28,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684/.tmp/info/a885d8ad6ba942fead6ef06a7b88c823 2023-02-08 23:56:28,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684/.tmp/info/a885d8ad6ba942fead6ef06a7b88c823 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684/info/a885d8ad6ba942fead6ef06a7b88c823 2023-02-08 23:56:28,314 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684/info/a885d8ad6ba942fead6ef06a7b88c823, entries=3, sequenceid=7, filesize=4.8 K 2023-02-08 23:56:28,315 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 31ce00ef22047cfd9f24d7e09ef0a684 in 413ms, sequenceid=7, compaction requested=false 2023-02-08 23:56:28,318 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/namespace/31ce00ef22047cfd9f24d7e09ef0a684/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-08 23:56:28,319 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:56:28,319 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 31ce00ef22047cfd9f24d7e09ef0a684: Waiting for close lock at 1675900587902Running coprocessor pre-close hooks at 1675900587902Disabling compacts and flushes for region at 1675900587902Disabling writes for close at 1675900587902Obtaining lock to block concurrent updates at 1675900587902Preparing flush snapshotting stores in 31ce00ef22047cfd9f24d7e09ef0a684 at 1675900587902Finished memstore snapshotting hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1675900587902Flushing stores of hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. at 1675900587903 (+1 ms)Flushing 31ce00ef22047cfd9f24d7e09ef0a684/info: creating writer at 1675900587903Flushing 31ce00ef22047cfd9f24d7e09ef0a684/info: appending metadata at 1675900587904 (+1 ms)Flushing 31ce00ef22047cfd9f24d7e09ef0a684/info: closing flushed file at 1675900587904Flushing 31ce00ef22047cfd9f24d7e09ef0a684/info: reopening flushed file at 1675900588312 (+408 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 31ce00ef22047cfd9f24d7e09ef0a684 in 413ms, sequenceid=7, compaction requested=false at 1675900588315 (+3 ms)Writing region close event to WAL at 1675900588316 (+1 ms)Running coprocessor post-close hooks at 1675900588318 (+2 ms)Closed at 1675900588319 (+1 ms) 2023-02-08 23:56:28,319 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675900417241.31ce00ef22047cfd9f24d7e09ef0a684. 2023-02-08 23:56:28,319 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b22d2ff0c5597d74f67d046e7f16564e, disabling compactions & flushes 2023-02-08 23:56:28,319 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:56:28,319 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:56:28,319 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. after waiting 0 ms 2023-02-08 23:56:28,319 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:56:28,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/b22d2ff0c5597d74f67d046e7f16564e/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-08 23:56:28,322 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:56:28,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b22d2ff0c5597d74f67d046e7f16564e: Waiting for close lock at 1675900588319Running coprocessor pre-close hooks at 1675900588319Disabling compacts and flushes for region at 1675900588319Disabling writes for close at 1675900588319Writing region close event to WAL at 1675900588320 (+1 ms)Running coprocessor post-close hooks at 1675900588322 (+2 ms)Closed at 1675900588322 2023-02-08 23:56:28,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1675900454803.b22d2ff0c5597d74f67d046e7f16564e. 2023-02-08 23:56:28,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 539644099f8843cf0be59ce746f2f9e3, disabling compactions & flushes 2023-02-08 23:56:28,323 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:56:28,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:56:28,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. after waiting 0 ms 2023-02-08 23:56:28,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:56:28,325 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/539644099f8843cf0be59ce746f2f9e3/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 23:56:28,326 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:56:28,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 539644099f8843cf0be59ce746f2f9e3: Waiting for close lock at 1675900588323Running coprocessor pre-close hooks at 1675900588323Disabling compacts and flushes for region at 1675900588323Disabling writes for close at 1675900588323Writing region close event to WAL at 1675900588323Running coprocessor post-close hooks at 1675900588325 (+2 ms)Closed at 1675900588326 (+1 ms) 2023-02-08 23:56:28,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1675900538480.539644099f8843cf0be59ce746f2f9e3. 2023-02-08 23:56:28,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 893b8071d052c13d0194a6716bc74a8c, disabling compactions & flushes 2023-02-08 23:56:28,326 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:56:28,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:56:28,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. after waiting 0 ms 2023-02-08 23:56:28,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:56:28,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/893b8071d052c13d0194a6716bc74a8c/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-08 23:56:28,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:56:28,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 893b8071d052c13d0194a6716bc74a8c: Waiting for close lock at 1675900588326Running coprocessor pre-close hooks at 1675900588326Disabling compacts and flushes for region at 1675900588326Disabling writes for close at 1675900588326Writing region close event to WAL at 1675900588327 (+1 ms)Running coprocessor post-close hooks at 1675900588329 (+2 ms)Closed at 1675900588329 2023-02-08 23:56:28,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675900418795.893b8071d052c13d0194a6716bc74a8c. 2023-02-08 23:56:28,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a9e06470572e5280a9822eb7682ca46a, disabling compactions & flushes 2023-02-08 23:56:28,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:56:28,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:56:28,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. after waiting 0 ms 2023-02-08 23:56:28,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:56:28,332 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithWeeklyPolicy/a9e06470572e5280a9822eb7682ca46a/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-08 23:56:28,333 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:56:28,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a9e06470572e5280a9822eb7682ca46a: Waiting for close lock at 1675900588329Running coprocessor pre-close hooks at 1675900588329Disabling compacts and flushes for region at 1675900588329Disabling writes for close at 1675900588329Writing region close event to WAL at 1675900588330 (+1 ms)Running coprocessor post-close hooks at 1675900588332 (+2 ms)Closed at 1675900588333 (+1 ms) 2023-02-08 23:56:28,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1675900444539.a9e06470572e5280a9822eb7682ca46a. 2023-02-08 23:56:28,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 777160b56e576a2f3a7f98e3a2aa2080, disabling compactions & flushes 2023-02-08 23:56:28,333 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:56:28,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:56:28,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. after waiting 0 ms 2023-02-08 23:56:28,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:56:28,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyPolicy/777160b56e576a2f3a7f98e3a2aa2080/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-08 23:56:28,337 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:56:28,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 777160b56e576a2f3a7f98e3a2aa2080: Waiting for close lock at 1675900588333Running coprocessor pre-close hooks at 1675900588333Disabling compacts and flushes for region at 1675900588333Disabling writes for close at 1675900588333Writing region close event to WAL at 1675900588334 (+1 ms)Running coprocessor post-close hooks at 1675900588336 (+2 ms)Closed at 1675900588337 (+1 ms) 2023-02-08 23:56:28,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1675900458424.777160b56e576a2f3a7f98e3a2aa2080. 2023-02-08 23:56:28,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 83e0134f0861e917b7a0acc88b520e50, disabling compactions & flushes 2023-02-08 23:56:28,337 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:56:28,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:56:28,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. after waiting 0 ms 2023-02-08 23:56:28,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:56:28,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/83e0134f0861e917b7a0acc88b520e50/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-08 23:56:28,341 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:56:28,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 83e0134f0861e917b7a0acc88b520e50: Waiting for close lock at 1675900588337Running coprocessor pre-close hooks at 1675900588337Disabling compacts and flushes for region at 1675900588337Disabling writes for close at 1675900588337Writing region close event to WAL at 1675900588339 (+2 ms)Running coprocessor post-close hooks at 1675900588341 (+2 ms)Closed at 1675900588341 2023-02-08 23:56:28,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1675900510544.83e0134f0861e917b7a0acc88b520e50. 2023-02-08 23:56:28,341 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a3bc650bd8f95f4b16a947f7e690d376, disabling compactions & flushes 2023-02-08 23:56:28,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:28,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:28,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. after waiting 0 ms 2023-02-08 23:56:28,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:28,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/a3bc650bd8f95f4b16a947f7e690d376/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 23:56:28,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:28,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a3bc650bd8f95f4b16a947f7e690d376: Waiting for close lock at 1675900588341Running coprocessor pre-close hooks at 1675900588341Disabling compacts and flushes for region at 1675900588341Disabling writes for close at 1675900588342 (+1 ms)Writing region close event to WAL at 1675900588343 (+1 ms)Running coprocessor post-close hooks at 1675900588345 (+2 ms)Closed at 1675900588346 (+1 ms) 2023-02-08 23:56:28,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1675900568720.a3bc650bd8f95f4b16a947f7e690d376. 2023-02-08 23:56:28,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2f876068b9705192eb71b2d2aa908931, disabling compactions & flushes 2023-02-08 23:56:28,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:56:28,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:56:28,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. after waiting 0 ms 2023-02-08 23:56:28,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:56:28,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2f876068b9705192eb71b2d2aa908931/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-08 23:56:28,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:56:28,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2f876068b9705192eb71b2d2aa908931: Waiting for close lock at 1675900588346Running coprocessor pre-close hooks at 1675900588346Disabling compacts and flushes for region at 1675900588346Disabling writes for close at 1675900588346Writing region close event to WAL at 1675900588347 (+1 ms)Running coprocessor post-close hooks at 1675900588349 (+2 ms)Closed at 1675900588349 2023-02-08 23:56:28,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675900487407.2f876068b9705192eb71b2d2aa908931. 2023-02-08 23:56:28,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6ef0d93c124a8d762314a9d7b3e5e1da, disabling compactions & flushes 2023-02-08 23:56:28,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:56:28,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:56:28,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. after waiting 0 ms 2023-02-08 23:56:28,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:56:28,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/6ef0d93c124a8d762314a9d7b3e5e1da/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-08 23:56:28,352 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:56:28,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6ef0d93c124a8d762314a9d7b3e5e1da: Waiting for close lock at 1675900588349Running coprocessor pre-close hooks at 1675900588349Disabling compacts and flushes for region at 1675900588349Disabling writes for close at 1675900588349Writing region close event to WAL at 1675900588350 (+1 ms)Running coprocessor post-close hooks at 1675900588352 (+2 ms)Closed at 1675900588352 2023-02-08 23:56:28,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1675900454803.6ef0d93c124a8d762314a9d7b3e5e1da. 2023-02-08 23:56:28,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 857c72d9be3c2ff762195748d825aa71, disabling compactions & flushes 2023-02-08 23:56:28,352 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:56:28,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:56:28,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. after waiting 0 ms 2023-02-08 23:56:28,353 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:56:28,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/857c72d9be3c2ff762195748d825aa71/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-08 23:56:28,356 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:56:28,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 857c72d9be3c2ff762195748d825aa71: Waiting for close lock at 1675900588352Running coprocessor pre-close hooks at 1675900588352Disabling compacts and flushes for region at 1675900588352Disabling writes for close at 1675900588353 (+1 ms)Writing region close event to WAL at 1675900588353Running coprocessor post-close hooks at 1675900588355 (+2 ms)Closed at 1675900588356 (+1 ms) 2023-02-08 23:56:28,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675900470761.857c72d9be3c2ff762195748d825aa71. 2023-02-08 23:56:28,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9d98bce82dc93804dedabab1ead06ebc, disabling compactions & flushes 2023-02-08 23:56:28,356 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:56:28,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:56:28,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. after waiting 0 ms 2023-02-08 23:56:28,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:56:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9d98bce82dc93804dedabab1ead06ebc/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 23:56:28,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:56:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9d98bce82dc93804dedabab1ead06ebc: Waiting for close lock at 1675900588356Running coprocessor pre-close hooks at 1675900588356Disabling compacts and flushes for region at 1675900588356Disabling writes for close at 1675900588357 (+1 ms)Writing region close event to WAL at 1675900588357Running coprocessor post-close hooks at 1675900588360 (+3 ms)Closed at 1675900588360 2023-02-08 23:56:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1675900538480.9d98bce82dc93804dedabab1ead06ebc. 2023-02-08 23:56:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ab31217c6a226641e72afe8624c897d8, disabling compactions & flushes 2023-02-08 23:56:28,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. after waiting 0 ms 2023-02-08 23:56:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:28,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMajorCompactionFromAdmin/ab31217c6a226641e72afe8624c897d8/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 23:56:28,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:28,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ab31217c6a226641e72afe8624c897d8: Waiting for close lock at 1675900588360Running coprocessor pre-close hooks at 1675900588360Disabling compacts and flushes for region at 1675900588360Disabling writes for close at 1675900588360Writing region close event to WAL at 1675900588361 (+1 ms)Running coprocessor post-close hooks at 1675900588363 (+2 ms)Closed at 1675900588363 2023-02-08 23:56:28,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1675900568720.ab31217c6a226641e72afe8624c897d8. 2023-02-08 23:56:28,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9246c8eb0029ec8f06f1acd496f0cde7, disabling compactions & flushes 2023-02-08 23:56:28,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:56:28,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:56:28,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. after waiting 0 ms 2023-02-08 23:56:28,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:56:28,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/ns/testMinorCompaction/9246c8eb0029ec8f06f1acd496f0cde7/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 23:56:28,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:56:28,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9246c8eb0029ec8f06f1acd496f0cde7: Waiting for close lock at 1675900588364Running coprocessor pre-close hooks at 1675900588364Disabling compacts and flushes for region at 1675900588364Disabling writes for close at 1675900588364Writing region close event to WAL at 1675900588365 (+1 ms)Running coprocessor post-close hooks at 1675900588367 (+2 ms)Closed at 1675900588367 2023-02-08 23:56:28,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1675900538480.9246c8eb0029ec8f06f1acd496f0cde7. 2023-02-08 23:56:28,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bf85808cb96a3fd70743466476b776c5, disabling compactions & flushes 2023-02-08 23:56:28,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:56:28,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:56:28,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. after waiting 0 ms 2023-02-08 23:56:28,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:56:28,372 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/bf85808cb96a3fd70743466476b776c5/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-08 23:56:28,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:56:28,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bf85808cb96a3fd70743466476b776c5: Waiting for close lock at 1675900588367Running coprocessor pre-close hooks at 1675900588367Disabling compacts and flushes for region at 1675900588367Disabling writes for close at 1675900588367Writing region close event to WAL at 1675900588370 (+3 ms)Running coprocessor post-close hooks at 1675900588372 (+2 ms)Closed at 1675900588373 (+1 ms) 2023-02-08 23:56:28,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1675900510544.bf85808cb96a3fd70743466476b776c5. 2023-02-08 23:56:28,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c091f7407da93dea8dbedd1c557bdac1, disabling compactions & flushes 2023-02-08 23:56:28,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:56:28,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:56:28,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. after waiting 0 ms 2023-02-08 23:56:28,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:56:28,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testScannerOnBulkLoadRefHFiles/c091f7407da93dea8dbedd1c557bdac1/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-08 23:56:28,375 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:56:28,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c091f7407da93dea8dbedd1c557bdac1: Waiting for close lock at 1675900588373Running coprocessor pre-close hooks at 1675900588373Disabling compacts and flushes for region at 1675900588373Disabling writes for close at 1675900588373Writing region close event to WAL at 1675900588373Running coprocessor post-close hooks at 1675900588375 (+2 ms)Closed at 1675900588375 2023-02-08 23:56:28,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1675900454803.c091f7407da93dea8dbedd1c557bdac1. 2023-02-08 23:56:28,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c0432a17915eb080602d94742abd61bf, disabling compactions & flushes 2023-02-08 23:56:28,375 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:56:28,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:56:28,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. after waiting 0 ms 2023-02-08 23:56:28,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:56:28,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testCompactionWithHFileLink/c0432a17915eb080602d94742abd61bf/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-08 23:56:28,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:56:28,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c0432a17915eb080602d94742abd61bf: Waiting for close lock at 1675900588375Running coprocessor pre-close hooks at 1675900588375Disabling compacts and flushes for region at 1675900588375Disabling writes for close at 1675900588375Writing region close event to WAL at 1675900588376 (+1 ms)Running coprocessor post-close hooks at 1675900588378 (+2 ms)Closed at 1675900588378 2023-02-08 23:56:28,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1675900510544.c0432a17915eb080602d94742abd61bf. 2023-02-08 23:56:28,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 56b1dbb83cb5199116335c5260eadbb4, disabling compactions & flushes 2023-02-08 23:56:28,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:28,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:28,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. after waiting 0 ms 2023-02-08 23:56:28,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:28,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/default/testMinorCompactionWithMonthlyPolicy/56b1dbb83cb5199116335c5260eadbb4/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-08 23:56:28,381 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:28,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 56b1dbb83cb5199116335c5260eadbb4: Waiting for close lock at 1675900588378Running coprocessor pre-close hooks at 1675900588378Disabling compacts and flushes for region at 1675900588378Disabling writes for close at 1675900588379 (+1 ms)Writing region close event to WAL at 1675900588379Running coprocessor post-close hooks at 1675900588381 (+2 ms)Closed at 1675900588381 2023-02-08 23:56:28,381 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1675900556148.56b1dbb83cb5199116335c5260eadbb4. 2023-02-08 23:56:28,491 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-08 23:56:28,691 DEBUG [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-08 23:56:28,712 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/.tmp/rep_barrier/09e13b823581463f8e021d47bf8be84d 2023-02-08 23:56:28,716 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09e13b823581463f8e021d47bf8be84d 2023-02-08 23:56:28,722 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/.tmp/table/ee9b23223efc40cab680f9a2487909d0 2023-02-08 23:56:28,725 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee9b23223efc40cab680f9a2487909d0 2023-02-08 23:56:28,726 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/.tmp/info/fe24e381d334492d9fa826c38e9e9201 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/info/fe24e381d334492d9fa826c38e9e9201 2023-02-08 23:56:28,730 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fe24e381d334492d9fa826c38e9e9201 2023-02-08 23:56:28,730 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/info/fe24e381d334492d9fa826c38e9e9201, entries=226, sequenceid=130, filesize=37.9 K 2023-02-08 23:56:28,731 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/.tmp/rep_barrier/09e13b823581463f8e021d47bf8be84d as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/rep_barrier/09e13b823581463f8e021d47bf8be84d 2023-02-08 23:56:28,734 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 09e13b823581463f8e021d47bf8be84d 2023-02-08 23:56:28,734 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/rep_barrier/09e13b823581463f8e021d47bf8be84d, entries=1, sequenceid=130, filesize=5.0 K 2023-02-08 23:56:28,735 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/.tmp/table/ee9b23223efc40cab680f9a2487909d0 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/table/ee9b23223efc40cab680f9a2487909d0 2023-02-08 23:56:28,738 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee9b23223efc40cab680f9a2487909d0 2023-02-08 23:56:28,738 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/table/ee9b23223efc40cab680f9a2487909d0, entries=27, sequenceid=130, filesize=6.6 K 2023-02-08 23:56:28,741 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 851ms, sequenceid=130, compaction requested=false 2023-02-08 23:56:28,747 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-08 23:56:28,747 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 23:56:28,747 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 23:56:28,747 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675900587889Running coprocessor pre-close hooks at 1675900587889Disabling compacts and flushes for region at 1675900587889Disabling writes for close at 1675900587889Obtaining lock to block concurrent updates at 1675900587889Preparing flush snapshotting stores in 1588230740 at 1675900587889Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51772, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1675900587889Flushing stores of hbase:meta,,1.1588230740 at 1675900587890 (+1 ms)Flushing 1588230740/info: creating writer at 1675900587890Flushing 1588230740/info: appending metadata at 1675900587894 (+4 ms)Flushing 1588230740/info: closing flushed file at 1675900587894Flushing 1588230740/rep_barrier: creating writer at 1675900588306 (+412 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675900588308 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675900588308Flushing 1588230740/table: creating writer at 1675900588716 (+408 ms)Flushing 1588230740/table: appending metadata at 1675900588718 (+2 ms)Flushing 1588230740/table: closing flushed file at 1675900588718Flushing 1588230740/info: reopening flushed file at 1675900588727 (+9 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675900588731 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1675900588735 (+4 ms)Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 851ms, sequenceid=130, compaction requested=false at 1675900588741 (+6 ms)Writing region close event to WAL at 1675900588745 (+4 ms)Running coprocessor post-close hooks at 1675900588747 (+2 ms)Closed at 1675900588747 2023-02-08 23:56:28,748 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-08 23:56:28,891 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42703,1675900414768; all regions closed. 2023-02-08 23:56:28,898 DEBUG [RS:0;jenkins-hbase4:42703] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/oldWALs 2023-02-08 23:56:28,898 INFO [RS:0;jenkins-hbase4:42703] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42703%2C1675900414768.meta:.meta(num 1675900417008) 2023-02-08 23:56:28,902 DEBUG [RS:0;jenkins-hbase4:42703] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/oldWALs 2023-02-08 23:56:28,902 INFO [RS:0;jenkins-hbase4:42703] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42703%2C1675900414768:(num 1675900416104) 2023-02-08 23:56:28,902 DEBUG [RS:0;jenkins-hbase4:42703] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:56:28,902 INFO [RS:0;jenkins-hbase4:42703] regionserver.LeaseManager(133): Closed leases 2023-02-08 23:56:28,902 INFO [RS:0;jenkins-hbase4:42703] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-08 23:56:28,902 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 23:56:28,903 INFO [RS:0;jenkins-hbase4:42703] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42703 2023-02-08 23:56:28,912 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42703,1675900414768 2023-02-08 23:56:28,912 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 23:56:28,912 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 23:56:28,913 ERROR [Listener at localhost/41521-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@12e4d9c0 rejected from java.util.concurrent.ThreadPoolExecutor@16db5f44[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:56:28,914 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42703,1675900414768] 2023-02-08 23:56:28,915 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42703,1675900414768; numProcessing=1 2023-02-08 23:56:28,917 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42703,1675900414768 already deleted, retry=false 2023-02-08 23:56:28,918 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42703,1675900414768 expired; onlineServers=0 2023-02-08 23:56:28,918 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39489,1675900413627' ***** 2023-02-08 23:56:28,918 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-08 23:56:28,918 DEBUG [M:0;jenkins-hbase4:39489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6fb60ee8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 23:56:28,918 INFO [M:0;jenkins-hbase4:39489] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39489,1675900413627 2023-02-08 23:56:28,918 INFO [M:0;jenkins-hbase4:39489] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39489,1675900413627; all regions closed. 2023-02-08 23:56:28,918 DEBUG [M:0;jenkins-hbase4:39489] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 23:56:28,918 DEBUG [M:0;jenkins-hbase4:39489] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-08 23:56:28,919 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-08 23:56:28,919 DEBUG [M:0;jenkins-hbase4:39489] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-08 23:56:28,919 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675900415750] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675900415750,5,FailOnTimeoutGroup] 2023-02-08 23:56:28,919 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675900415750] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675900415750,5,FailOnTimeoutGroup] 2023-02-08 23:56:28,919 INFO [M:0;jenkins-hbase4:39489] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-08 23:56:28,919 INFO [M:0;jenkins-hbase4:39489] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-08 23:56:28,919 INFO [M:0;jenkins-hbase4:39489] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-08 23:56:28,919 DEBUG [M:0;jenkins-hbase4:39489] master.HMaster(1502): Stopping service threads 2023-02-08 23:56:28,920 INFO [M:0;jenkins-hbase4:39489] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-08 23:56:28,920 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-08 23:56:28,920 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 23:56:28,920 ERROR [M:0;jenkins-hbase4:39489] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-08 23:56:28,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 23:56:28,920 INFO [M:0;jenkins-hbase4:39489] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 23:56:28,921 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-08 23:56:28,921 DEBUG [M:0;jenkins-hbase4:39489] zookeeper.ZKUtil(398): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-08 23:56:28,921 WARN [M:0;jenkins-hbase4:39489] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-08 23:56:28,921 INFO [M:0;jenkins-hbase4:39489] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 23:56:28,921 INFO [M:0;jenkins-hbase4:39489] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-08 23:56:28,921 DEBUG [M:0;jenkins-hbase4:39489] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 23:56:28,921 INFO [M:0;jenkins-hbase4:39489] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:56:28,921 DEBUG [M:0;jenkins-hbase4:39489] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:56:28,921 DEBUG [M:0;jenkins-hbase4:39489] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 23:56:28,921 DEBUG [M:0;jenkins-hbase4:39489] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:56:28,921 INFO [M:0;jenkins-hbase4:39489] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.77 KB heapSize=598.48 KB 2023-02-08 23:56:28,928 INFO [M:0;jenkins-hbase4:39489] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.77 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6fcffbde5db2426198717eb4a5f44f84 2023-02-08 23:56:28,931 INFO [M:0;jenkins-hbase4:39489] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fcffbde5db2426198717eb4a5f44f84 2023-02-08 23:56:28,931 DEBUG [M:0;jenkins-hbase4:39489] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6fcffbde5db2426198717eb4a5f44f84 as hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6fcffbde5db2426198717eb4a5f44f84 2023-02-08 23:56:28,934 INFO [M:0;jenkins-hbase4:39489] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fcffbde5db2426198717eb4a5f44f84 2023-02-08 23:56:28,934 INFO [M:0;jenkins-hbase4:39489] regionserver.HStore(1080): Added hdfs://localhost:36245/user/jenkins/test-data/b0b3309e-d647-45b0-76d8-26b74d2e97d7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6fcffbde5db2426198717eb4a5f44f84, entries=123, sequenceid=1005, filesize=33.3 K 2023-02-08 23:56:28,935 INFO [M:0;jenkins-hbase4:39489] regionserver.HRegion(2947): Finished flush of dataSize ~512.77 KB/525081, heapSize ~598.47 KB/612832, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 14ms, sequenceid=1005, compaction requested=false 2023-02-08 23:56:28,936 INFO [M:0;jenkins-hbase4:39489] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 23:56:28,936 DEBUG [M:0;jenkins-hbase4:39489] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675900588921Disabling compacts and flushes for region at 1675900588921Disabling writes for close at 1675900588921Obtaining lock to block concurrent updates at 1675900588921Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675900588921Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=525081, getHeapSize=612832, getOffHeapSize=0, getCellsCount=1160 at 1675900588922 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675900588922Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675900588922Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675900588925 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675900588925Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675900588932 (+7 ms)Finished flush of dataSize ~512.77 KB/525081, heapSize ~598.47 KB/612832, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 14ms, sequenceid=1005, compaction requested=false at 1675900588935 (+3 ms)Writing region close event to WAL at 1675900588936 (+1 ms)Closed at 1675900588936 2023-02-08 23:56:28,938 INFO [M:0;jenkins-hbase4:39489] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-08 23:56:28,938 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 23:56:28,939 INFO [M:0;jenkins-hbase4:39489] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39489 2023-02-08 23:56:28,941 DEBUG [M:0;jenkins-hbase4:39489] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,39489,1675900413627 already deleted, retry=false 2023-02-08 23:56:29,014 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:56:29,014 INFO [RS:0;jenkins-hbase4:42703] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42703,1675900414768; zookeeper connection closed. 2023-02-08 23:56:29,015 ERROR [Listener at localhost/41521-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@56decf20 rejected from java.util.concurrent.ThreadPoolExecutor@16db5f44[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:56:29,015 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): regionserver:42703-0x100365acd470001, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:56:29,015 ERROR [Listener at localhost/41521-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2e2d6481 rejected from java.util.concurrent.ThreadPoolExecutor@16db5f44[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:56:29,015 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7e62358d] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7e62358d 2023-02-08 23:56:29,015 INFO [Listener at localhost/41521] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-08 23:56:29,114 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:56:29,115 INFO [M:0;jenkins-hbase4:39489] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39489,1675900413627; zookeeper connection closed. 2023-02-08 23:56:29,115 ERROR [Listener at localhost/41521-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6190ecc0 rejected from java.util.concurrent.ThreadPoolExecutor@32e52bca[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:56:29,115 DEBUG [Listener at localhost/41521-EventThread] zookeeper.ZKWatcher(600): master:39489-0x100365acd470000, quorum=127.0.0.1:54774, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 23:56:29,115 ERROR [Listener at localhost/41521-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@54517039 rejected from java.util.concurrent.ThreadPoolExecutor@32e52bca[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 23:56:29,116 WARN [Listener at localhost/41521] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 23:56:29,120 INFO [Listener at localhost/41521] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 23:56:29,227 WARN [BP-834230988-172.31.14.131-1675900410558 heartbeating to localhost/127.0.0.1:36245] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-08 23:56:29,227 WARN [BP-834230988-172.31.14.131-1675900410558 heartbeating to localhost/127.0.0.1:36245] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-834230988-172.31.14.131-1675900410558 (Datanode Uuid 837303ae-269b-4144-a273-fa40483f1512) service to localhost/127.0.0.1:36245 2023-02-08 23:56:29,230 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/cluster_0361639e-2335-beea-2849-07295ee226f0/dfs/data/data2/current/BP-834230988-172.31.14.131-1675900410558] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 23:56:29,230 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/530b6bb0-8533-3578-7bda-306413f3845d/cluster_0361639e-2335-beea-2849-07295ee226f0/dfs/data/data1/current/BP-834230988-172.31.14.131-1675900410558] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 23:56:29,258 INFO [Listener at localhost/41521] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 23:56:29,390 INFO [Listener at localhost/41521] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-08 23:56:29,472 INFO [Listener at localhost/41521] hbase.HBaseTestingUtility(1293): Minicluster is down